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ABSTRACT 

The dissemination of research findings concerning 
postsecondary education governance and finance is considered in the 
proceedings of a 1986 conference sponsored by the National Center for 
Postsecondary Governance and Finance. Included are four commissioned 
papers, a background paper, a paper reviewing the role of 
effectiveness, and an overview of participant reaction to these 
papers. Paper titles and authors are as follows: "Planning for 
Research Use in Postsecondary Governance and Finance" (Ward S. 
Mason); "On Learning To Add" (William N. Dunn); "Effective 
Dissemination in a National Education Research Center" (John H. 
Hollifield); "Meshing the Gears: Reporting on Higher Education in 
America" (Mike Bowler); "The Consensus Development Conferences of the 
National Institutes of Health as a Model for Education and Other 
Fields" (Ronald G. Havelock); and "Dissemination, Communication 
Effects and Organizational Effectiveness" (Kathryn Theus). Included 
is a framework for disseminating policy rebearch that includes policy 
analysis, technical communication, and public information. Mr. 
Hollifield* s paper describes dissemination activities of The Johns 
Hopkins Center for Effective Elementary and Middle Schools. (SW) 
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FOREWORD 



Since its formation in 1985, the National Center for Postsecondary 
Governance and Finance, a higher education research center funded by 
OERI/OR, has pursued a systematic process of planning for the dissemination 
of its research. Dissemination is a major function of all research centers 
sponsored by the Department of Education's Office of Educational Research 
and Improvement (OERI/OR). Dissemination is especially important in a center 
focusing on policy research, since policy research inherently is concerned with 
applied issues. 

Ideas for dissemination were solicited from a variety of sources during 
the process of writing the technical application for the center grant, and the 
application contained the outline of the Center's initial program. During the 
first year of operation, relatively routine dissemination activities cofncided 
with the planning of a more comprehensive program. These activities included 
preparation of a brochure, two newsletters, press releases concerning the 
Center's establishment and program, and presentations at a number of 
professional meetings and conferences. 

A planning meeting held in August 1986, brought together a number of 
higher education representatives, who helped structure a dissemination 
conference to guide Center planning for future years. (See Appendix A for a 
list of attendees.) At the conference held on Nov. 21, 1986 (see list of 
participants in Appendix B), four individuals made presentations on facets of 
the dissemination function based on thair own background and experience. The 
four included: Mike Bowler, a journalist with the Baltimore Evening Sun and 
president of the Education Writers Association; William N. Dunn, professor and 
director of the Study of Knowledge Use Program, Graduate School of PubHc 
and International Affairs, University of Pittsburgh; John Hollifield, assistant 
director of the Center for Effective Elementary and Middle Schools, The Johns 
Hopkins University; and Ronald G. Havelock, research professor at the Center 
for Productive Use of Technology, George Mason University. The four 
commissioned papers presented by these individuals constitute the core of the 
present Proceedings . General editor, Ward Masor/s May 1986 pre-conference 
background paper, as well as a paper reviewing the role of effectiveness 
prepared by Kathryn Theus, Certer Associate Director, are also included. A 
group of dissemination specialists served as respondents to these presentations. 
James Bencivenga, director of the Information Services Program and Salvatore 
Corrallo, Center Liaison, represented the Center's sponsor, OERI/OR. Four 
members of the Center staff also attended. Mason chaired the discussion 
following the presentations. 
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Summary of Conference Discussions 



Richard Chait, executive director of the Center, welcomed and introduced 
the visitors. To provide a framework for considering dissemination issues, he 
indicated his view of the Center's target audiences in rank order, 

o campus leaders (presidents, trustees, etc), 

o state policy makers, 

o federal policy makers, 

o the scholarly community, and 

o the lay community 

These stakeholders vary considerably in their interests and concerns, Chait 
pointed out that the Center needs to consider the extent to which each 
requires different products and approaches. 

In a broader context, the Center also needs to be wary of several aspects 
of the current environment. One is the demand for simple answers to complex 
problems. While simple answers may not be available, there is still a need to 
provide understandable summaries and interpretations. Information, also should 
be provided to focus public attention on areas of crisis or popular concern. 

The Center knows how to disseminate its findings to the research 
community. It is less certain about identifying effective strategies for 
informing other constituencies. 

Frank Schmidtlein, the Center's associate director for research, expressed 
the desire to keep expectations for dissemination efforts realistic. At the 
elementary/secondary levels, the Office of Educational Research and 
Improvement (OERI) supports a larger number of research centers, a network 
of regional educational laboratories, and specialized dissemination programs 
such as the National Diffusion Network (NDN). In higher education there are 
only two centers and no equivalent laboratories. However, laboratories do not 
limit themselves to pre-college education. Several have a particular interest in 
the articulation of research between pre-college and higher education. At all 
levels the Educational Resource Information Center (ERIC) serves as access 
points to research findings and provides complementary services. 

Salvatore Corrallo, OERI/OR liaison to the Center, underscored the need 
to plan for dissemination from each project's beginning and to shape 
dissemination efforts to reach particular audiences. 

James Bencivenga, director of the Information Services Program of 
OERI/OR, presented a federal perspective on the issue. An overarching goal 
of OERI/OR is to promote an intellectual exchange across all levels of society. 
Responding to Chait's listing of target audiences, he suggested attention to 
governors and their aides. 

In the area of higher education finance, OERI/OR has a great interest in 
actual costs, particularly in the difference between public and private 
institutional costs, and who bears the costs. 

2 
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As a journalist, Bowler believes that when disseminating through the mass 
media, comparisons are the primary way of communicating complex findings. 
For example, comparing our educational system with that of Japan and of 
European countries is very helpful. The Department of Education is about to 
release a report on Japanese education that should have quite an impact. 
Another enlightening comparison might include our educational system with 
corporate classrooms. 

In the background paper, Mason noted that the Center's commitment to 
policy research implies a strong interest in research application and utilization. 
Yet the historical record on the use of research in policy making is ambiguous 
at best and discouraging at worst. This suggests the need to be clear about 
what is meant by use and to develop different approaches for working with 
policy makers at different stages of the policy-making process. 

Several "general principles 0 were propounded: 

o Planning dissemination should not be put off until "later when we 
have some research completed; 1 ' concern for research use should 
enter into Center thinking as research is being planned. 

o The Center should not think of itself as a lone operator, responsible 
for every step of a long-term programmatic effort. Rather, it is a 
part of several communities with which it can work out useful 
divisions of labor. 

o There tends to be a negative relationship between the scope of 
diffusion activities and the strength of their impact. 

o The Center should give full consideration to all three major modes 
of dissemination: information services, professional development, and 
technical assistance. 

o The Center should attempt to base its dissemination plans on the 
findings of research utilization studies, and it should commit some of 
its resources to making a contribution to knowledge in that field. 

Mason presented a framework for structuring a dissemination plan 
consisting of the following categories: 



Iil connection with each category the following questions should be addressed: 



o 



o 



o 



o 



o 



Planning 

Public information 
Staff services 
Outreach 

Research/evaluation 



o 



o 



o 



What activities should the Center undertake? 
What are the staffing implications? 

How can the Center make maximum use of its resources by 
working with and through other organizations? 
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Placement of Papers in a Conceptual Framework 

In introducing the papers featured at the conference, and the analysis by 
Theus, it is helpful to place them within a framework; (presented in Figure 1) 
which is structured around the need to develop different dissemination 
approaches Cor different audiences. The three major audiences — policy 
makers, researchers and lay stakeholders are shown on the right. Obviously- 
these publics might be more finely differentiated, but they serve to highlight 
the major distractions. Each of the papers focus on reaching different 
audiences, often by similar means. 

William N. Dunn writes from the perspective of both a theorist and one 
who manages a program that provides policy analysis to the Pennsylvania State 
Legislature. His paper, "On Learning to ADD," provides a set of conceptual 
tools for understanding the relationship between policy research and policy 
making, for developing appropriate information products, and for disseminating 
them to policy makers. For him, the key to this is refining the nature of 
policy analysis, which he differentiates from policy research. Policy analysis \s 
a practical methodology for dealing with the problems of specific clients in a 
short time frame using readily available sources of information. Its methods 
include sound logical reasoning, comparison of pro and con arguments, and 
simple arithmetic. Typical products including policy issue papers, policy 
memoranda, and policy news releases. Methods of disseminating policy analysis 
documents are very pragmatic and feature many forms of personal contact (see 
Fig- 1). 

Figure 1 

Framework For Disseminating Policy Research 
Dissemination Method Publics 



POLICY ANALYSIS 
(See Dunn & Have lock) 
o Policy issue paper o 
o Policy memorandum o 
o Policy brief o 
o Policy news release o 

o 
o 

TECHNICAL COMMUNICATION 
(See Hollifield & Havelock) 
o Technical reports o 
o Journal articles o 
o Books and Chapters o 



PUBLIC INFORMATION 
(see Bowler & Havelock) 
o Brochure o 
o Newsletter o 
o Inquiries o 
o Press releases 



POLICY MAKERS 

Transmitting 

Forwarding 

Calling 

Meeting 

Briefing 

Conferencing 

RESEARCHERS 

Publishing 
Conferencing 
Networking with 
"Invisible colleges" 

STAKEHOLDERS 

Publishing 

Meeting 

Replying 



ERIC 



4 



John HollifielcTs paper, "Effective Dissemination in a National 
Education Research Center," describes dissenination activities of The 
Johns Hopkins Center for Effective Elementary and Middle Schools. He 
makes an important distinction betwsc.x research findings and research 
products. 

To disseminate research findings , the Hopkins Center uses a variety 
of publications, including a raport series, journal articles, books, book 
chapters, conference papers, and input to the ERIC system. The Hopkins 
Center also develops prototypic products — "more specific sets of 
processes and/or materials for direct use by practitioners to improve their 
schools and classrooms." Products are disseminated through commercial 
publication, the National Diffusion Network, the dissemination programs of 
the OEPI/OR sponsored educational laboratories, state education agency 
dissemination projects, and through other outlets. 

In disseminating to the research community, Hollifield places 
emphasis on providing full reports that permit an examination foi 
theoretical, methodological, and interpretive accuracy and completeness. 
In disseminating to policy makers and practitioners, who generally have 
neither time nor inclination to delve into full reports, emphasis is placed 
on providing summaries and interpretative articles that spell out research 
implications for specific audiences and particular problems. For all 
projects, dissemination accomplishments are documented through a 
dissemination profile which lists the types of information, types of 
channels, types of audiences reached, the potential number of persons 
reached, and the timing of the dissemination activity. 

In his paper, "Meshing the Gears: Reporting on Higher Education in 
America," Mike Bowler provides the interesting perspective of a member 
of the worki g press and president of the Education Writers Association. 
His remarks are directed largely at the public information section of Fig 
1. 

Bowler notes that people are interested in education news, for it 
involves two things they value intensely, their children and their money. 
However, he warns that education reporting is a low status beat, and only 
a handful of reporters specialize in higher education. Bowler also 
stresses that different audiences have different needs and are reached 
through different media, including: 

o Higher education journals such as the Chronicle and Change, 
o National media such as The New York Times , Newsweek , and 

the networks, 
o Local daily papers. 

To effectively use the mass media for dissemination, Bowler 
emphasizes the need to understand the "culture" of the newsroom. 
Reporters are almost all generalists, interested in stories with the local 
angle. They like comparisons among cities and states and quantitative 
data such as SAT scores. They still want "man bites dog" stories, and 



graphics and commission reports can have appeal. The Center needs to 
have personal contacts within each medium. The press thrives on 
controversy, so take advantage of the interest that controversy can 
arouse* Good clear writing is, of course, essential. 

Ronald G. Havelock's paper, "The Consensus Development 
Conferences of the NIH as Model for Education and Other Fields' 1 focuses 
on a very specific dissemination model. The Consensus Development 
Model he discusses has potential relevance to both policy analysis and 
technical communication (see Fig. 1), by bringing the research and policy 
making communities together. 

The purpose of the NIH consensus conferences has been to achieve 
expert agreement where a significant amount of research is available. 
Other purposes have included: 1) building bridges between clinical 
research specialists and primary care physicians, 2) understanding 
technology assessment, 3) diffusing medical knowledge to the general 
public, and 4) articulating new research agendas. 

Havelock describes the model in some detail and then explores 
whether conditions that seem to support the success of the model in 
health fields also obtain in education. He finds that the higher education 
community is analogous in some ways to the medical community. Both 
adhere to collegial, rather than bureaucratic norms. Education, like 
agriculture, has attempted to formalize the process of research diffusion 
and to study how it works. By contrast, medicine has given relatively 
little attention to diffusion efforts and processes. 

The keynote paper by Mason, the four conference papers and the 
paper by Theus are presented in the following sections, followed by an 
overview cf participant reaction to these papers. 



- Ward S. Mason 
August, 1987 
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PLANNING FOR RESEARCH USE 
IN POSTSECONDARY GOVERNANCE AND FINANCE 



The National Center for Postsecondary Governance and Finance is 
now midway into its first year of operation. It is important that it firm 
up its approach to providing information and services to its constituencies 
and to developing a leadership role in its mission area, both for the 
balance of this first year and in the longer term of its five-year grant. 

There is some tendency to consider dissemination as a minor activity 
of a research center. However, careful consideration of the many facets 
of the problem of research utilization suggests that dissemination should 
be accorded status as a major function of an educational R&D center. 

The fact that the National Center for Postsecondary Governance and 
Finance views its mission as focused on policy research is of special 
significance. This means that the Center intends to judge itself 
ultimately on the usefulness of its research to policy-makers. Granted 
that basic research is often an essential basis for creative approaches to 
policy issues, the Centers mission is fundamentally an applied mission. 

Thus, we are concerned with the whole issue of the relation of 
research to practice. A great deal has been learned about this in the 
past 10 to 20 years. Indeed, a whole new research specialty focusing on 
research utilization has emerged. It has its own journal - Knowledge: 
Creating, Diffusion, and Utilization - and professional networks (e.g. the 
Special Interest Group on Research Utilization of AERA). Research 
syntheses have begun to appear (Human Interaction Research Institute, 
1976; Havelock, 1969; and Lehming and Kane, 1981). Work has been done 
in a number of economic sectors. Although research in education has 
been done primarily at the elementary/secondary level, there is good 
reason to believe that many findings have potential applicability to higher 
education as well. 

A broad approach to dissemination is advised* We are concerned not 
only with the provision of information about the results of research, but 
also with a wide range of other mechanisms for the transfer of 
knowledge, plus a variety of factors which facilitates or hinders the use 
of knowledge. 

A number of writers have documented the apparently disappointing 
record on the use of formal policy analysis for the formulation and 
implementation of policy (Mitchell, 1981; Lindblom and Cohen, 1979; Lynn, 
1978). Perhaps that disappointment is valid if one expects a simple 
instrumental relationship between analysis and action. But there are 
many meanings of research use (Weiss, 1979), and more often research 
serves an enlightenment function. 
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Research does not solve problems; it provide evidence that can 
be used by men and women of judgment in their efforts to 
reach solutions* It helps to establish the premises on which 
the debate shall take place, and a conceptual base for the 
discussion of policy (Weiss, 1978, 76-77), 

Others have stressed that policy-making is a process, not an event 
(Mitchell, 1980). Knowledge is useful in different ways at the different 
stages of the process. At the stage of defining the issues, conceptual 
and theoretical perspectives may be of primary interest. At the policy 
deliberation stage, the results of research studies are relevant. In the 
policy implementation stage, implementation research comes into play. 
Policy-making, in contrast to policy analysis, is interactive and thus 
subject to political and other considerations, not just the canons of 
rationality (Lindblom and Cohen, 1979). 

General Principles 

As the Center approaches its dissemination planning task, it should 
follow several general principles: 

1. Planning dissemination should not be put off until "later when 
we have some research completed;" concern for research use should enter 
into Center thinking and planning from the beginning. Some questions 
need to be addressed: 

o Who are the major audiences and constituencies for the 

Center's research? 
o What are the major policy issues in governance, management, 

and finance being faced by the higher education community? 
o What degree of impact does the Center hope to achieve in each 

program area vith each constituency? Awareness? 

Experimentation with change? Institutionalization of change? 
o What responsibilities for dissemination functions should 

different staff members have? For which functions are 

dissemination specialists needed and which should be among the 

responsibilities of research personnel? 
o In which networks and communities should the Center 

participate? 

2. The Center should not think of itself as a lone operator, 
responsible for every step of a long term programmatic effort. Rather, it 
is part of several, including both the higher education and educational 
research communities. It needs to seek points of leverage with which it 
can extend its impact. This can be done by working cooperatively with 
other organizations and by sharing resources. 

3. There tends to be a negative relationship between scope of 
diffusion and strength of impact (Louis et. al; 1984, p. 234). Given the 
wide variety of potential constituencies, the Center needs to face these 
trade-offs and decide which to target for impact and which will be 
satisfied with awareness or intermediate levels of dissemination. 

9 
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4. The Center should give full consideration to all three major 
goals of dissemination (Mason, 1982). The term "dissemination" is a 
difficult one because it tends to connote only the provision of 
information services . There are two other personnel-intensive modes that 
need to be considered. Professional development uses people to organize 
and convey knowledge in terms of the generic problems of specific 
positions (e.g. deans of academic units). The Center may offer both staff 
development and graduate training activities that fit this mode. Technical 
assistance uses people to organize and convey knowledge, or to implement 
specific innovations. This is an expensive mode of dissemination for 
which Center resources may not be adequate. However, such programs 
can be very rewarding by providing "reality testing" of knowledge 
generated by the research program. The Center should give careful 
consideration to providing such services on at least a limited basis. 
Costs might be minimized by providing the services on a contract or fee 
basis. 

5. The Center should base its dissemination plans on the findings 
of research utilization studies, and it should commit some of its resources 
to making a contribution to knowledge concerning research utilization, 
both the practical knowledge needed for its own operation and basic 
knowledge of more general interest. Examples: 

o Research methods, among others, should be used to 
systematically identify the policy issues that are salient to 
policy makers in different roles and institutional settings. 

o Systematic data should be collected on dissemination programs 
and activities in order to build a stronger knowledge base for 
what works best under what circumstances. Part of the 
Center's evaluation plan that focuses on dissemination should 
be designed both to be of practical assistance to the Center 
and to make a contribution to the field of research utilization. 

Note that every center and laboratory has a responsibility for 
dissemination of its R&D. The study of research use should be a prime 
area for cooperative activity since it is of common interest and has few 
barriers to cooperation. 

Major Dissemination Activities 

Major activities concerned with research use can be divided into 
five general categories as follows: 

o Planning 

o Public Information 

o Staff Services 

o Outreach 

o Research/Evaluation 

Planning . Dissemination activities of the Center should operate 
under a dissemination plan , updated yearly. This should be part of the 

10 



operating plan submitted to the Office of Educational Research and 
Improvement/Office of Research (OERI/OR), 



Public Information . Any publicly funded research center assumes an 
obligation to explain its mission and describe its activities to the public 
and its various constituencies. To this end, the Center should publish an 
attractive brochure and make it available to visitors and at conferences 
and meetings where presentations are made. Communication channels to 
the press should be maintained, and press releases should be issued to the 
educational and general press describing newsworthy reports or events. 
The capability for giving attention to foreign and domestic visitors should 
be established. 

These public information functions are straightforwarded and 
presumably non-controversial. Two others are somewhat problematic, A 
national center naturally attracts a number of inquiries. Handling some 
of these is simply a fact of organizational life, e,g, inquiries concerning 
the Center itself, or the sorts of queries that normally find their way 
into professor offices. Given that the Center is a national project it 
might be natural to assume that it has an obligation to act as a general 
information center in its mission area. However, there is no merit in 
duplicating the function now filled by the ERIC system or other services. 

Similarly, the publication of a newsletter would seem to be a natural 
development. It may well be warranted, but before this step is taken 
other alternatives should be carefully explored. For example, it may be 
possible to have regular columns or inserts in the newsletters of other 
publications of higher education associations. These different releases 
could be more narrowly directed than could a newsletter mailed to the 
varied constituencies of the Center, This would be a good topic to take 
up with the National Advisory Panel, 

Staff Services . The Center should have specialized personnel who 
can provide support services in the following areas: 

Publication s 

o Design and operate of publications program, including books, 
monographs, journal articles, occasional papers, newsletters, 
etc, 

o Propose publication policies, e.g, what should be published 
directly by the Center and what should be published elsewhere, 
o Provide editing and graphics services, 
o Maintain appropriate mailing lists. 

Meetings 

o Develop logistics for internal and external meetings (e,g. 
National Advisory Panel, Technical Advisory Groups, staff 
development workshops, etc). 

o Prepare and staff displays and presentations at professional 
conferences, 

11 
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Reference 



o Organize a reference room and a minimal body of reference 
materials in the Center's mission area, but use other 
institutional resources wherever possible. 

Electronic networking 

o Maintain an electronic bulletin board linking Center staff in its 
four locations, 

o Facilitate Center participation in electronic networks 
maintained by OERI/OR and professional associations, as 
appropriate. 

Reporting 

o Assist the Associate Director for Research in monitoring the 
timely submission of reports and deliverables to OERI, 

Cooperative Arrangements 

o Develop and maintain cooperative arrangements with other 
organizations that can help the Center disseminate its results, 
including ERIC, the Institute for Educational Leadership, higher 
education associations, and others who disseminate information 
about higher education. 

o Assist research staff in participating in cooperative activities 
with other OERI supported centers and laboratories. 

Technical assistance to researchers on designing dissemination 
strategies and materials 

o Participate with the Associate Director of Research in project 
reviews to ensure they provide for products and processes that 
will facilitate effective dissemination of research results and 
assure that dissemination issues are brought before the RCC, 



Outreach . The core of a dissemination plan concerns choices to be 
made with respect to providing information and services to various 
constituencies. The specifics are dependent on the Center's view of its 
mission and the nature of its research program. The Center's 
programmatic "turf" may be described in terms of a six-cell table as 
follows: 
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Program Areas 



Stimulus Governance Management Finance 



National Trends and 
External Agencies 
Affecting Campuses 

Campus Concerns 



Higher Education 
Governing Theory and 
Ethical Issues 



Programmatic efforts might be described in terms of the columns, the 
rows, or the cells. (Not all cells have entries for the first year.) If, in 
any of these cells clusters of projects have sufficient coherence, then 
they might also provide a thematic structure for the design of outreach 
dissemination activities. Such coherence does not always obtain the 
Center's efforts are essentially project-driven. 

The Center's program plan makes an important distinction between 
two major audiences or constituencies, i.e. policy makers and researchers. 
In this paper, the plan's treatment of the traditional modes of 
communication within the research community — publications, 
conferences, and networking — are adequate and best handled by research 
personnel of each project. The discussion of the Center's mission of 
serving poliy-makers is organized under three headings: needs 
assessment, professional development, and technical assistance. 

Needs Assessment . A research program tends to "work forward"; a 
dissemination programs needs to be "planned backwards" (cf "backward 
mapping" in Elmore, 1979-80). The idea is to start by defining the needs 
of the constituency to be served and then ask what should be done to 
meet those needs. 

A first step is to segment the audience more carefully. The term 
"policy-makers" covers a lot of territory. It includes people in many 
different roles, institutional settings, and levels of the higher education 
system. The distinction between policy-makers per se and 
"administrators" is important. The former implies focus on choice among 
policy alternatives, while the latter suggests the need to be brought up- 
to-date on the latest in knowledge or conceptual perspectives to assist in 
managing current activities. Given the need to serve a broad range of 
policy-makers, it might be useful to select a few subgroups (e.g. 
presidents of research universities or chairs of state higher education 
legislative committees) for more intensive effort. 
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Secondly while the Center seems to define its mission largely in 
terms of policy research, study of the project descriptions reveals little 
attention to clear-out policy issues. Most projects indicate that they will 
address "issues such as . . .", and most of the reports will apparently be 
primarily descriptive. Systematic efforts to define the policy issues of 
interest to specific groups within one or more of the nine cells should be 
undertaken. This would lead to a sharpening of the policy issues or 
information needs to be addressed in specific projects. 

The Center has recognized the importance of the National Advisory 
Panel and the various Technical Advisory Committees in fulfilling this 
needs assessment function. Needs assessment should be part of the 
perspective of all staff whenever they meet with outside groups. 

Professional Development . The Center has made a general 
commitment to developing curricula for staff development programs and 
for pre-service graduate programs, and a few projects have plans that fall 
in this area. It is appropriate to work out the details of a professional 
development plan by addressing such questions as the following: 

o What do needs assessments (see above) indicate as primary 
areas for training in governance, management, and finance? 

o What cooperating institutions and associations are interested in 
working with the Center to develop curricula and offer courses 
and workshops? 

o Can specific types of products be designed which would bring 
together the results of projects in thematic areas, e.g. a 
reader, synthesis papers or volumes, etc. 

o How might modern audio-visual media be utilized to up-grade 
curricula? 

Technical Assistance . The Center has few, if any, plans for 
providing technical assistance to policy makers. As noted above, the 
personnel costs of such assistance are high, although these might be 
reduced by offering the assistance on a contract or fee basis. Given the 
applied nature of the Center's mission, some work of this kind is highly 
recommended in order to provide "hands-on" experience in the issues 
being studied. Probably the project directors and their associates for 
specific projects would be in the best position to provide technical 
assistance. The time is probably not yet ripe for developing specialized 
technical assistance staff such as are found in elementary/secondary 
education. 

Two projects that would seem to lend themselves to playing a 
technical assistance role are those on Statewide Board Evaluation and 
Institutional Planning, but others might be identified as well. 
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Research/Evaluation , As noted above, the Center should attempt to 
base its dissemination plan on the findings of utilisation research and 
that it should commit some resources to adding to this body of 
knowledge* It already has the responsibility for evaluating its own 
program. Dissemination research might best be handled as part of the 
evaluation function, for dissemination research can be viewed as the 
attempt to evaluate the effectiveness of dissemination strategies. Practical 
kinds of data needed for managing the dissemination program (e.g. 
questionnaires on reactions of planning workshop participants) might be 
considered part of the formative evaluation function. Field work which 
addresses more basic questions of research use might be undertaken as 
part of the summative evaluation function. 

Allocation of Responsibilities for Dissemination 

The activities and tasks outline above are quite varied both in 
nature and skill level required. No doubt as a whole they are beyond the 
cluster of talents likely to be found in any one individual. Therefore, it 
is necessary to think in terms of allocating the responsibilities between 
specialized dissemination positions and research pcritions, and between 
regular staff and outside or occasional personnel. It is important to 
recognize that dissemination in an important institutional function for 
which all staff have some responsibility. For those tasks, particularly in 
the public information area, that require mid-level personnel, the Center 
might well share these positions with other research or administrative 
units of the School of Education. Presumably the Associate Director for 
Dissemination and other specialized staff would be located at the 
University of Maryland. Undoubtedly research staff at other sites will 
have access to editorial, reference, and other services. 

In recruiting an Associate Director for Dissemination there are two 
options. One would be to define the job in terms of a "vertical cluster" 
of tasks: essentially those listed in the irst column of Fig. 2. Under 
this option the Center would want to recruit someone with a broad range 
of communication skills who was willing to work at both higher and lower 
skill levels. Under this option, the research staff takes a fairly heavy 
responsibility for dissemination. 

The alternative would be to define the job in terms of a more 
"horizontal cluster." In this case, the Associate Director for 
Dissemination might take on more of the responsibilities which have been 
assigned to research staff in Fig. 2. This option probably requires that 
some of the more routine tasks be assigned to other mid-level staff. 
This may be quite feasible in a university setting where there is the 
possibility of sharing such people with other departments and centers. 
However, under this option it would be necessary to find an individual 
whose background included not only communication but also strong 
substantive interest in higher education governance and some knowledge 
of social science research. 
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Figure 2 

ALLOCATION OF DISSEMINATION RESPONSIBILITIES 

POSITION 

Associate Research Consul- Evalu- 

Director Staff tation uation 

Team 



Planning X 

Public Information 

o Center brochure X 

o Press release X 

o Visitor services X 
o Answering public 

inquiries X 

o Newsletter X 

Staff Services 

o Publications X 

o Meetings X 

o Reference X 

o Electronic Network X 

o Reporting X 

Cooperative Arrangements 

o For Research Y 

o For dissemination X 
o Tech Assistance 

for researchers Y 



Outreach 
o 
o 



Needs Assessment 
Professional 
Development 
a. Staff DeveJL 
be Graduate Educ. 
o Tech. Assistance 
Research/Evaluation 

o Practical/formative 
o Besic/summative 



X 
Y 



Y 
Y 



X 
X 
X 



Y 
Y 



Key: X = primary responsibility 



Y = secondary responsibility 
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ON LEARNING TO ADD 



The primary mission of the National Center for Postsecondary 
Governance and Finance is to condrct research which is useful to 
colleges and universities* To be genuinely useful such research must 
illuminate or alleviate practical problems faced by college and university 
administrators, their faculties, and other stakeholders in the success of 
higher education. For example: 

o What strategies of educational planning have been most 
successful in improving the cost-effectiveness of academic 
programs? Do these strategies work equally well in small and 
large institutions? What strategies are most appropriate in a 
particular context? 

o How can colleges and universities develop more effective forms 
of collaboration with industry? Do university-industry 
partnerships facilitate advances in scientific R&D, while at the 
same time creating new sources of income? What form of 
collaboration is best suited to a particular context? 

o Is the termination of general revenue sharing likely to create 
large deficits for state-supported colleges and universities? 
What is the probable magnitude of these deficits in different 
institutions? Which compensatory financial options are most 
appropriate under particular circumstances? 



In principle, these and similar practical problems may be illuminated 
or alleviated through directed research* ^o achieve this end, however, it 
is not sufficient simply to conduct high-quality research* High-quality 
research also must contribute to the problem-solving capacities of colleges 
and universities* 1 

Analysis 

The problem-solving capacities of colleges and universities can be 
enlarged through two modes of inquiry* One mode is educational policy 



*An extensive literature demonstrates that good research does not 
necessarily contribute solutions for real-life problems* See, for example, 
C.E. Lindblom and D. Cohen, Usable Knowledge (New Haven: Yale 
University Press, 1979); C*H* Weiss, Social Science Research , and Decision 
Making (New York: Columbia University Press, 1980); G* Beal et* ah , 
Knowledge Generation, Exchange, and Utilization (Boulder: Westview 
Press, 1986). 
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analysis, while the other is education policy research. 2 Contrasts 
between these two modes of inquiry are essential to the Center's present 
and evolving problem-solving role. 

o Source of Problems . Policy analysis illuminates or alleviates 
problems faced by a specific client (e.g., the president of a 
rural community college). Policy research addresses problems 
faced by general classes of potentially interested clients (e.g., 
major research universities). 

° Audience . Policy analysts communicate findings to a specific 
audience (e.g., the special assistant to a college president). 
Policy researchers communicate with a general audience (e.g., 
an association of college admissions officers or editors, 
reviewers, and readers of a professional journal). 

o Time Frame . The time-frame of policy analysts may range 
from several weeks to months (e.g., a prompt response may be 
essential to brief administrators prior to annual legislative 
appropriations). Policy research has a time-frame ranging from 
several months to years (e.g., a book, monograph, or report 
typically requires three years from inception to publication and 
distribution). 

o Scope . In policy analysis, observations are based on the 
synthesis of existing sources of information (e.g., NCES data 
archives, periodicals, key informants). Policy research requires 
observations based on new sources of information (e.g., 
ethnographic or sample survey data). 

o Methods . Policy analysis involves sound logical reasoning, 
comparison of contending arguments, and simple arithmetic 
(e.g>, a well-reasoned narrative backed up by tables). Policy 
research, by contrast, involves a range of complex methods 
(e.g., benefit-cost analysis, causal modeling, linear 
programming). 

o Products . The typical products of policy analysis are the 
policy issue paper, policy options paper, policy memorandum, 
and policy news release. The typical products of policy 
research are the research report, research monograph, scholarly 
book, and journal article. 



2 The distinction between policy research and policy analysis is based 
on P.J. Cook and J.W. Vaupel, "What Policy Analyst Do: Three Research 
Styles," Journal of Policy Analysis and Management 4.3 (Spring 1985), 
427-28; and W.N. Dunn, "If Educational Policy Analysis Is the Solution, 
What is the Problem? Options for Regional Labs," Southeast Educational 
Improvement Laboratory, Research Triangle Park, North Carolina, March, 
1987. 
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Policy analysis has a greater likelihood of making specific, direct, 
and immediate contributions to problem-solving capacities of colleges and 
universities. These contributions may range from improving conceptual 
capacities to formulate and diagnose problems to improving instrumental 
capacities to identify, develop, and implement specific new policies. 3 
Policy research has a greater likelihood of making general, indirect, and 
delayed contributions to problem-solving capacities of colleges and 
universities. These contributions also may be conceptual as well as 
instrumental. 

Development 

Educational policy analysis is but the first step in improving 
problem-solving capacities. Analysts also must communicate the results of 
their inquiries, which demands skill in the development of policy-relevant 
documents. Among the most important of these documents are: 

o The Policy Issue Paper . This is the most general of policy- 
relevant documents produced by an analyst. Policy issue papers 
provide answers to one o: # more of the following questions: In 
what different ways may the policy problem be defined? What 
is the scope and severity of the problem? How is the problem 
likely to change in future months or years? What goals and 
objectives should be pursued to solve the problem? What major 
policy alternatives should be considered as potential solutions 
for the problem? To what extent have previous efforts to 
solve the problem been successful? What credible policy 
options are available? What options should be recommended? 
Analysts are rarely requested to answer all these questions. 
Instead, they are typically asked to provide answers to more 
limited questions; for example, questions involving the 
formulation of a problem, the evaluation of past policy 
performance, or the comparison of available policy options. 

o Policy Memoranda . Policy memoranda draw upon the 
conclusions and recommendations of a policy issue paper. 
Sometimes policy memoranda respond to or challenge these 
conclusions and recommendations, or their assumptions. 
Lengthy policy memoranda are similar to policy issue papers; 
only the form and style of communication differ. Short policy 
memoranda are useful for communicating the major conclusions 
or recommendations of a policy issue paper, but without 
providing the full text. 



3 In other words, policy analysis may perform "conceptual" as well as 
"instrumental" functions which have been distinguished at length in 
studies of research utilization. For a review of these and other 
distinction see W.N. Dunn, "Measuring Knowledge Use," Knowledge: 
Creation, Diffusion, Utilization 5,4 (1983): 120-133. 
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o Policy Briefs . These are abbreviated versions of full policy 
issue papers* Essentially, a policy brief is an extended 
executive summary, often including key references, but 
excluding appendices* Policy briefs are useful and appropriate 
when the analyst wishes to communicate a concise summary of 
the larger policy paper, but without providing the full 
document. 

o Policy News Release . In dealing with representatives of the 
mass media, analysts are frequently asked to provide a concise 
summary of the major conclusions or recommendations of a 
policy issue paper. Such requests are frequently the product of 
initiatives taken by clients who contact the media in order to 
publicize the contents of a paper. A policy news release may 
assume the form of an abbreviated executive summary. For 
example, a policy news release may identify the author(s) and 
client(s) of a policy issue paper and describe only the 
recommendations included in an executive summary. In cases 
where the policy issue paper and executive summary have other 
goals — for example, to describe policy options — the news release 
would contain conclusions rather than recommendations. 



The several variants of policy-relevant documents described above 
call attention to the fact that there are many audiences for the same 
policy-relevant information. Clients are frequently only one audience; and 
effective communication with different audiences may require different 
policy-relevant documents. 4 For example, policy news releases are most 
appropriate for representatives of the mass media and public interest 
groups. Executive summaries and policy briefs, by contrast, are more 
suitable for communicating with policymakers and other analysts. The 
appropriateness of a policy-relevant document is governed by the 
characteristics of particular segments of the audience and their 
informational needs. 

The development of policy-relevant documents requires different 
kinds of skills than • hose needed to conduct policy analysis. The 
development of policy-relevant documents requires skills in synthesizing 
policy-relevant information; organizing the written text into logically 
connected sections; translating specialized and sometimes esoteric 
language into the varied languages of clients and other stakeholders; 
simplifying complex processes of scientific and technical reasoning into 
clearly stated arguments; displaying essential information in the form of 
tables, graphs, and figures; summarizing key background information, 
conclusions, and recommendations in the form of an executive summary; 
and appending the most relevant support documents to the text of the 
paper. 



4 See A. Meltsner, Policy Analysts in the Bureaucracy , Rev. Ed. 
{Berkeley: University of California Press, 1986). 
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Dissemination 



Although different educational policy documents may be developed 
for different audiences, these documents must be disseminated to intended 
beneficiaries. Just as the process of analyzing policies is different from 
developing documents, so is the process of disseminating these documents 
different from both. The process of dissemination involves: 

o Transmitting . Policy-relevant documents are transmitted to 
clients via letters of transmittal. 

o Forwarding . Copies of policy-relevant documents are forwarded 
to other stakeholders. 

o Calling . Clients and other stakeholders are contacted by 
telephone in order to clarify or call attention to documents. 

o Meeting . Clients and other stakeholders are met face-to-face 
to clarify or call attention to documents. 

o Briefing . Clients and other stakeholders are briefed in 
meetings scheduled to present the contents of documents. 

o Conferencing . Clients and other stakeholders confer in meeting 
schedules to exchange and discuss the contents of documents. 

The availability of advanced computer technology and satellite 
communication makes it possible to engage in the above activities as part 
of computer networks. For example, requests for policy-relevant 
information, together with responses in the form of policy papers, are 
transmitted between policymakers and analysts via telecomputer networks. 
Electronic mail and electronic bulletin boards are replacing standard 
letters, memoranda, telephone calls and meetings, while interactive tele- 
briefings and tele-conferences are broadcast via microwave and satellite 
communications. Analysts are now able to access computerized data bases 
which contain policy papers and reports, model legislation, and statistical 
series useful for conducting policy analyses at the national, state, and 
local levels. While there is no doubt about the rapid development of new 
communications technologies, it is not yet clear whether these 
technological advances are improving the conduct of policy analysis and 
the development and dissemination of policy-relevant documents. 

Although there are no specific rules or criteria for effective 
dissemination, we do have some general guidelines: 

o Audience Segmentation . Although we cannot anticipate all 
relevant stakeholders who will be or should be part of the 
audience, thinking strategically about the composition of the 
audience is essential for effective communication. There are 
many stakeholders to be reached* Effective disse?mination 
demands audience segmentation. 
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o Contextual Mapping , Each segment of the audience is likely to 
differ in ways that will affect the dissemination of policy 
analysis. Particular audience segments tend to have distinctive 
frames of reference, that is, organized sets of assumptions and 
standards of assessment which shape their perception and 
evaluation of policy-relevant information. Policy-relevant 
documents should be responsive to these competing criteria. 

o Translation . The language of policy analysis is not necessarily 
the language of the audience. Different audience segments 
with different reference frames use contrasting languages. 
Analytical and technical terms may be fully appropriate for 
audiences who are producers or regular consumers of 
professional policy analysis. But other audiences require more 
readily understood terms and clear explanations or analytic and 
technical terms. 

o Product Differentiation . Policy-relevant products may be 
differentiated according to the audience, its particular 
interpretive structure, and its distinctive language. Multiple 
documents may be necessary for different audience segments. 
Product differentiation involves the development of documents 
with alternative formats and lengths — for example, a full policy 
paper as contrasted with a news release. Tne language, style, 
and types of arguments also may be adapted to a particular 
audience without compromising the methodological and 
substantive integrity of a document, including its conclusions 
and recommendations. 

o Medium Differentiation . Media for disseminating policy- 
relevant documents may also be differentiated. Simply 
transmitting a document may be insufficient; telephone 
conversation, face-to-face meetings, formal briefings, and 
conference are also important communications media. The 
establishment of computer networks which permit electronic 
mail, electronic bulletin boards, and interactive computer 
conferencing are similarly important as alternative media. 

Learning to (A)nalyze, (D)evelop, and (D)isseminate 

The aim of educational policy analysis is to improve educational 
policies. But good policy analysis does not speak for itself; it must be 
developed into policy-relevant documents and disseminated to various 
audiences before it can contribute to the problem-solving capacities of 
colleges and universities. Although it would be highly useful to have a 
well-tested and codified body of rules for performing analysis, 
development, and dissemination, no such body of established rules 
presently exists. Indeed, analysts, developers, and disseminators are in 
much the same position as clinical practitioners within such older 
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professions as clinical psychology and psychotherapy. 5 Educational policy 
analysis and other problem-solving disciplines have yet to develop a sound 
theory of applications which would specify the conditions under which 
analysis are most likely to illuminate or alleviate practical problems. 6 

Under these conditions, where no well-tested and codified body of 
rules is available for improving practice, we must be content with 
inductive learning strategies which permit continuous assessments of the 
practical impact of analysis, development, and dissemination. The logic 
and procedures of quasi-experimentation are particularly well-suited to 
this task, since quasi-experimentation is expressly designed as an 
inductive learning strategy in complex practice settings where simple 
deductive theories are inapplicable. 7 

Any particular effort to analyze, develop, and disseminate may be 
regarded as a practical experiment and represented as the functional 
equation 

PI = f(A, D, D) + Error 

where PI represents a presumed practical impact, A represents a 
particular form of policy analysis, D represents a variation in a policy- 
relevant document, and (the second) D represents a particular 
dissemination strategy. Error stands for that part of PI which cannot be 
accounted for or explained by A, D, and D. 

An expanded model of relationships between analysis, development, 
and dissemination is presented in Figure 1. The ADD Model explicitly 
incorporates analysis, development, and dissemination as key manipulatable 
features of practical problem-solving. The ADD Model also draws 
attention to the quintessential practical aim of analysis, development, and 
dissemination, which is to learn how to add to problem-solving capacities. 



5 See B. Fischoff, "Clinical Policy Analysis, 11 in W.N. Dunn (ed.) 
Policy Analysis: Perspectives, Concepts, and Methods (Greenwich, CT: 
JAI Press, 1986). 

6 See C.H. Weiss, Social Science Research and Decision Making , Ch. 1 
and W.N. Dunn, "Evaluating the Effects of Policy Analysis: Toward a 
Theory of Applications," in S.S. Nagel (ed.) Public Policy Analysis and 
Management (Greenwich, CT: JAI Press, 1986). 

7 See T.D. Cook and D.T. Campbell, Quasi-Experimentation (Boston: 
Houghton Mifflin, 1979). 
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FIGURE 1 
THE ADD KODEL 
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We must recognize that the practical impact (PI) of analysis (A), 
development (D), and dissemination (D) is likely to be ambiguous, 
uncertain, or equivocal* This equivocality is due in part to 
characteristics of analysis (A), which involves variations between and 
within producers, products, and procedures* In order to infer that an 
observed practical impact, PI, is a consequence of A, policy analysis, we 
must be able to learn which features of policy analysis are causally 
efficacious* For example, PI may be due to: 

o Producer Variability* The analytic styles of policy analysts 
vary* Some analysts are "technicians" while others are 
"politicians" or "entrepreneurs*" 8 

o Product Diversity * The products of policy analysis may range 
from memoranda and policy briefs to lengthy policy papers, 
reports, and studies* Although policy analysis is supposed to 
yield recommendations for actions, available evidence suggests 
that the content of products varies widely* For example, 
books, articles, and reports produced by a sample of thirty 
policy-oriented research institutes display two markedly 
different orientations, the "scientific" and "humanistic*" 9 A 
content analysis of 181 randomly sampled articles published in 
in leading policy journals from 1975 through 1980 suggests, 
among other findings, that products display two broadly 
divergent orientations: "quantitative-empirical" and "rhetorical- 
discussive*" 10 

o Procedural Variability * Procedures employed to conduct policy 
analysis are also variable* There is little working consensus 
among practitioners about the procedures that are most 
appropriate: 11 nor is there sufficient evidence to show that any 
preferred set of procedures (e.g., cost-benefit analysis) is 
faithfully applied in contexts of practice where practitioners 
must perform a variety of essential non-analytic functions: 
instill confidence in clients, ask questions to obtain sensitive 
or controversial information, manage conflicts, and crises, 
understand what is not being said, avoid the imposition of the 



8 See Meitsner, Policy Analysts in the Bureaucracy . 

9 Pamela Doty, "Values in Policy Research," in W.N. Dunn (ed.) 
Values, Ethics, and the Practice of Policy Analysis (Lexington: D.C. 
Heath, 1983). 

10 J.A. Schneider. N.J. Stevens, and L.G. Tornatzky, "Policy Research 
and Analysis: An Empirical Profile, 1975-1980," Policy Sciences 15 (1982): 
99-114. 

n G.D. Garson, "Policy Science: A Quarter Center of Progress?" in 
Dunn (ed.) Policy Analysis: Perspectives, Concepts, and Methods 
(Greenwich, CT: JAI Press, 1986). 
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analyst's own values and perceptions, and devise strategies for 
interactive problem solving. 12 

The process of disseminating policy-relevant products is also 
variable. While a linkage system unites producers and users, 13 the aims, 
attitudes, and perceptions of users are seldom congruent with those of 
producers. The professional frames of referenc3 of users and producers 
often diverge, one consequence of which is the division of the linage 
system into two insulated communities. 14 Differences in professional 
reference frames reflect conflicting standards for assessing the truth, 
relevance, and cogency of knowledge gained through policy analysis. 15 
These conflicting standards of assessment appear to be correlated with 
differences in the structure of communications networks and the 
distribution of bureaucratic power and the types of strategies chosen to 
communicate knowledge. 16 

An essential measure of practical impact (PI) is whether results of 
policy analysis have been utilized* Although the utilization of policy- 
relevant knowledge is frequently treated as a discrete choice involving a 
decision to accept or reject the conclusions or recommendations contained 
in a policy-relevant document, the process of knowledge utilization is 
complex. The utilization of policy analysis has at least three major 
dimensions: 17 

o Composition of Utilization . Policy analysis is utilized by 
individuals as well as organizations. The concept of an 



12 Fischhoff, "Clinical Policy Analysis," p. 35. 

13 See R.G. Havelock, Planning for Innovation (Ann Arbor: Institute 
for Social Research, 1973). 

14 N. Caplan, "The Two-Communities Theory and Knowledge 
Utilization," American Behavioral Scientist 22 (1979): 459-70. 

15 See, e.g., Holzner and E.M. Fisher, "Knowledge-in-Use: 
Considerations in the Sociology of Knowledge Applications," Knowledge: 
Creation, Diffusion, Utilization 1 (1979): 219-44; C.H. Weiss and M.J. 
Bucuvalas, "Truth Tests and Utility Tests: Decision Makers' Frames of 
Reference for Social Science Research," American Sociological Review 45 
(2980): 302-12; W.N. Dunn, "Reforms As Arguments," Knowledge: 
Creation, Diffusion, Utilization 3 (1982): 293-326; and D.T. Campbell, 
"Experiments As Arguments," Knowledge: Creation, Diffusion, Utilization 
3 (1982): 327-41. 

16 See, respectively, E.M. Rogers and D.L. Kincaid, Communications 
Networks (New York: Free Press, 1981); and R.F. Rich, Social Science 
Information and Public Policy Making (San Francisco: Jossey-Bass, 1981). 

17 Dunn, "Measuring Knowledge Use." 
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"improved decision" implies that utilization is a process of 
individual decision making. By contrast, the concept of 
utilization as "enlightenment" or "collective learning" affirms 
that the utilization of policy analysis is a process, 

o Expected Effects of Utilization . The utilization of policy 
analysis has mental as well as behavioral effects. Distinctions 
between these two kinds of eifects captures contrasts between 
"conceptual," utilization, on one hand, and "instrumental" 
utilization on the other. Conceptual utilization refers to 
change in the ways that individual and collectives think or feel 
about practical problems and their solutions-. Instrumental 
utilization, by contrast, denotes observable changes in 
individual or collective actions, for example, changes in a 
legislators' publicly espoused position on an issue or changes in 
the allocation of resources in a university. Thus, the expected 
effects of utilization may be individual or collective in 
composition. 

o Scope of Utilization . The utilization of policy analysis may be 
specific or general. The concept of "ideas in good currency" 18 
is general in smpe, while the concept of adopting a policy 
recommendation is specific. The scope of utilization may be 
viewed ao gradients along a scale which range from being 
aware of or acquainted with conclusions, options, or 
recommendations to being able to explain them to others or put 
them into effect. 19 The same* scope of utilization, whether 
general or specific, may be individual or collective in 
composition and have* conceptual or behavioral effects. 

These three intersecting dimensions — composition, expected effect, scope — 
create new opportunities to assess the practical impact of analysis, 
development, and dissemination. 

Returning to the sources of variability discussed above [PI = f (A, D, 
D,) + Error] we can now specify potential errors committed when we 
assess the practical impact of analysis, development, and dissemination. 
These errors occur when an observed practical impact is falsely attributed 
to analysis, development, or dissemination when, instead, some other 
factor is responsible for the observed practical impact (see Figure 2). 



18 See D.A. Schon, The Reflective Practitioner (New York: Basic 
Books, 1983). 

19 See F. Machlup, Knowledge: Its Creationi Distribution and 
Economic Significance (Princeton: Princeton University Press, 1980). 
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FIGURE 2 



SOURCES OF ERROR IN ASSESSING PRACTICAL EFFECTS 
OF ANALYSIS, DEVELOPMENT, AND DISSEMINATION 

Error 



Methods-in-Use . Methods actually employed by analysts, apart from those 
which they formally espouse, provide alternative explanations or practical 
impacts. For example, the putative practical success of cost-benefit 
analysis is actually due to the analyst's skills in managing conflict and 
building collaborative relationships with policymakers. 

Multiple Analytic Interference . Multiple analytic procedures are jointly 
applied, making inferences about practical impact atypical of the separate 
application of a given procedure. For example, no practical impact is 
observed when analysts employ multiple methods, even though a single 
method may have resulted in a positive effect. 

Materials Bias . The sequence of presenting materials, the format for 
their presentation, or the process of translating methods and conclusions, 
rather than the conclusions of policy analysis, provide alternative 
explanations of practical impact. For example, the translation of 
scientific concepts into ordinary knowledge may govern practical impacts. 

Solution-Regression . A policy analysis is often commissioned when 
problems are most severe ("th<? problem must get worse before it gets 
better"), creating pressures for solution that do not depend on policy 
analysis. Analysis may be an effect of a policy problem which is so 
severe that it is bound to improve without analysis. 

Analytic Structure . The structure of analytic roles, rather than methods 
and conclusions, provides an alternative explanation of practical impact. 
For example, the influence of "entrepreneurs," rather than the substantive 
conclusions reached through analysis, explains practical impact. 

Multiple Interpretation Interference . The interpretations of producers and 
u^ers are sufficiently different that practical impacts cannot be separated 
from their interpretation. For example, weak or non-existent impact may 
be due to conflicting reference frames for interpreting analyses, not to 
conclusions of analyses themselves. 

Latent Function . The latent functions of policy analysis provide 
alternative explanations of impact. For example, latent functions of 
political control, political ammunition, or program subversion, rather than 
the conduct of analysis, explain practical impact. 
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Linkage Structure . The pattern of linage roles, dissemination routines, 
and institutional incentives, rather than the conduct of analysis, provide 
alternative explanations of impact. For example, the incentive system of 
analysts may be oriented towards professional rewards, while the 
incentive system of policymakers may be oriented towards institutional 
survival. 

Dimensionality Bias. One or more dimensions of utilization (e.g., 
composition) are confounded with another dimension (e.g., expected 
effects or scope), making inferences about practical impact atypical of the 
separate occurrence of a single dimension. For example, practical impact 
may be assessed in terms of expected effects which are specific in scope, 
leading to claims that no practical impact occurred, when in fact 
utilization did occur in the form of general effects. 



SOURCE: Adapted from Dunn, "Assessing the Impact of Policy Analysis: 
Toward a Theory of Applications" (1986). 



Conclusion 



There is no well-tested and codified body of rules for doing 
educational policy analysis and developing and disseminating documents 
based on results on this analysis. Under these conditions the best we 
can do is to use quasi-experimentation and other inductive learning 
strategies which permit us to learn from our errors. But we need to be 
able to know an error when we see one, relating this knowledge to 
variations in analysis, development, and dissemination. Otherwise learning 
will be random, not systematic; it also will be costly. 

Learning to ADD (analyze, develop, disseminate) requires a 
willingness to confront the kinds of errors described above. Most 
importantly, it requires an acknowledgement that we do not know — except 
in the most general and obvious sense — how to conduct research and 
analysis which illuminates or alleviates practical problems of 
postsecondary education. 
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EFFECTIVE DISSEMINATION IN A NATIONAL 
EDUCATION RESEARCH CENTER 



The effective dissemination of research findings and research 
products is obviously a major concern for a national education research 
center. It is a concern that the Johns Hopkins Center for Research on 
Elementary and Middle Schools has been grappling with for about 20 
years (previously as the Johns Hopkins Center for Social Organization of 
Schools). 

Over this length of time, the Center has employed a large number of 
dissemination outlets to try to get its research findings and products into 
use by schools and districts and into used by education policymakers. We 
have, as time progressed, incorporated these strategies and outlets into a 
general Dissemination Plan — a categorization of the methods and the 
outlets that are generally U3ed to disseminate Center research findings 
and products. 

Part I of this paper presents this Dissemination Plan. It describes 
the dissemination methods and channels use by the Hopkins Center and 
the rationale for their use, and provides some examples of their 
effectiveness. It will not, of course, be directly transferable to the needs 
of the Maryland Center, but should provide some assistance in developing 
dissemination strategies and an effective dissemination program. 

Part II of this paper comments on a variety of the issues faced by a 
research center in the initial stages of establishing effective dissemination 
policies, These comments represent "craft knowledge" — opinions based 
on experience — and are presented in a Question-Discussion format. 

The Hopkins Center Dissemination Plan 

The Center has a strong background in dissemination and puolic 
information and has developed, over the years, a consistent set of 
institutional activities that are applied to each research project and 
tailored to meet each project's potential for dissemination to researchers, 
practitioners, and policy makers in education. 

The Center's work produces two types of materials that require 
dissemination — research findings and research produc's. Findings are 
information and new knowledge derived from research which can be used 
by others to enhance their own research, make policy decisions, improve 
their school practices, and so on. All Center research projc ,s produce 
findings to be disseminated. Some Center projects, based on their 
findings, proceed to develop prototypic products — more specific sets of 
processes and/or materials for direct use by practitioners to improve their 
schools and classrooms. Thus the Center's dissemination activities focus 
on two areas: (A) dissemination of research findings, and (B) 
dissemination of research products. 
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Dissemination of Research Findings 



A national survey of school uses of microcomputers produces reams 
of pertinent data about the numbers of micros in schools, who's using 
them, how they're being used, what effects they're having on students, 
A re-analysis of a time-on-task equals greater achievement for most 
students. Another study finds that students who follow a "New Basics" 
curriculum in high school show greater achievement - but only if they're 
high achievers to begin with. 

These are few of the research findings in recent Center studies. 
They have implications that bear directly on the work of two general 
audiences — education researchers, and education practitioners, including 
policy makers. The general dissemination plan of the Center incorporates 
specific mechanisms to reach each of these audiences. 

Dissemination to the research community . Research findings need to 
be disseminated to other researchers who can react to the findings, 
incorporate them in their own research, replicate or refine them, or 
challenge them, and generally build upon them to eventually produce a 
coherent set of knowledge in a particular topic area. To accomplish this, 
researchers need full reports of the research to examine carefully for 
theoretical, methodological, and interpretive accuracy and completeness. 
The Center disseminates full reports of its research to social scientists 
and education researchers through direct mailing, journal publication, 
monographs, book chapters, books, research conferences, and the ERIC 
system. 

o Research findings are reported first in the Center Report 
Series. Each report describes the study conducted, the method, 
the results, and the implications of the results for further 
research and educational practice. The reports are mailed to a 
targeted mailing list of: a) preeminent educational, 
psychological, and sociological researchers; b) OERI and other 
Department of Education personnel; c) Maryland and Baltimore 
City education research personnel and administrators; d) the 
network of educational laboratories and research centers and 
other education research institutions; and e) large educational 
research library departments. More than 350 Center 
educational research studies have been disseminated through 
this series, 

o Research findings are published in scientific journals such as 
American Educational Research Journal , Journal of Educational 
Psychology , Sociology of Education , Review of Educational 
Research , Elementary School Journal , and many other refereed 
professional journals. This dissemination outlet is probably the 
most effective for reaching other researchers and influencing 
the conduct and direction of educational research. It is a key 
aspect of scientific communication. 
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o When program research produces a significant body of studies 
or work in a specific area, the Center seeks commercial 
publication in book form to disseminate the information. Books 
authored or edited by Center researchers have been published 
by Academic Press, Longman, D,C, Heath/Lexington, NEA 
publishing, Prentice-Hall, Plenum, and The Johns Hopkins 
University Press, Center researchers have achieved commercial 
publication of sixteen books, most of which are syntheses and 
summaries of original research accomplished in a specific area, 

o Research findings are prepared by Center researchers for 
publication as chapters in commercial textbooks, annual 
compilations, literature syntheses, and so on, and more than 50 
such chapters have been published in the past five years, 

o Research findings are presented at national and regional 
conferences of researchers and practitioners. Center staff 
present their studies annually at conventions of the American 
Educational Research Association, American Sociological 
Association, and the American Psychological Association; and 
also deliver invited presentations each year at local, regional, 
national, and international conferences. 

Presentations at these conferences allows Center researchers to 
reach and influence a large number of other researchers and 
practitioners whose interests lie in similar areas. In turn, the 
Center research is influenced by the work of other presenters. 
Thus presentation at these conferences and attendance at other 
conference sessions is often analogous to holding a national or 
regional conference on a specific theme without the expense 
that such a conference would entail. This is especially true of 
symposia designed by Center researchers and presented at 
AERA, APA, and ASA each year, 

o The Center places its research reports into the ERIC system 
for dissemination. The importance of the ERIC system as a 
dissemination mechanism for education research is often 
underestimated. The system is a major source of research 
information for many groups, and often the only source 
consulted, ERIC searches of the literature are undertaken by 
researchers seeking information on research topics of interest, 
and by administrators and students in education courses seeking 
information about a problem area. Through ERIC, the Center 
research findings are brought to the attention of these groups. 
Also, Center reports entered into ERIC contain full references 
that direct users of these reports to other journal articles and 
books of interest. 

Dissemination to practitioners and policy makers . Education 
practitioners and policy makers have neither the time, inclination, nor 
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technical interest to delve into full reports of research studies and their 
findings. For this audience, summaries and syntheses of research findings 
are disseminated through news releases and articles for the education 
media and professional and association publications. In addition, specific 
interpretive articles that report the findings more extensively and 
interpret their implications for the specific audience are prepared for 
these publications, either by Center staff or by editors assigned by the 
publications. The Center also disseminates interpretation and implications 
of its research through its own Report , which goes to over 8,000 
education personnel ranging from district superintendents to school of 
education deans, and through the publication of targeted newsletters on 
specific topics ( School Uses of Microcomputers , Summary of Teacher 
Practices of Parent Involvement , STL Newsletter ). Also, the Center 
works closely with the ERIC Clearinghouse on Elementary Education to 
produce summaries of its research findings in appropriate formats for the 
Clearinghouse. 

The breadth of this dissemination activity, which is conducted for 
each Center research project, is best illustrated by an example — the 
dissemination of research findings for the School Uses of Microcomputers 
project, which began in early 1982. Dissemination activities have included 
responding to over 1,500 requests for technical research reports; 
production of 11 journal articles and a book chapter; a monograph 
published by the International Council for Computers in Education; 
inclusion of the research in the ERIC system; articles written specifically 
for the ERIC Clearinghouse MicroNotes newsletter; conference 
presentations at AERA; invited presentations at the National Educational 
Computing Conference, Computers in Education '83, the World Conference 
of Computers in Education, a Phi Delta Kappan Conference; publication of 
research findings in Education Daily , ERS Bulletin , PTA Today , Education 
Week , Education Times , Report on Education Research , Nation's Schools 
Report , Education Marketer , in various state and local education 
newsletters, and in Time Magazine , The New York Times , Washington Post 
and numerous other large-city newspapers; appearances on CBS News and 
the PBS series on educational computing; and extensive interpretive 
articles published in NASSP Bulletin , Classroom Computer Learning , 
College Board Review , ETS Focus Magazine , Creative Computing , 
Electronic Learning , and others. In addition, two extensive articles were 
published in the Center's Report, and a special communications project 
was developed ( School Uses of Microcomputers Newsletter ) to report 
research findings from the project as they developed. The initial mailing 
list of the Newsletter was approximately 6,000 subscribers, due to 
requests to be put on the mailing list. 

The dissemination activity for each project is documented by a 
Dissemination Profile which lists the types of information disseminated, 
the types of media, the specific media, the types of audiences reached, 
the potential number of persons reached, and the timing of the 
dissemination activity. 

These diverse dissemination activities are conducted for all Center 
research projects, and produce extensive coverage of research findings 
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that are of interest to and have practical uses for practitioners. This 
activity functions as a communication system that increases awareness of 
the value of research to educators and the general public and provides 
the information to specific audiences who will be able to use it. The 
effectiveness of these dissemination activities is illustrated by the 
Center's record of publication of research findings with five of the major 
general education media publications: Education Daily (and its affiliates), 
Education Week, Education Times , Education USA , and ERS Bulletin . In 
1983-84, reports of our research findings appeared in those publications 
32 times. In addition, in 1983-84, 20 reports and interpretive articles of 
our research findings appeared in targeted association publications ( ASCD 
Update , NASSP News Leader , AFT American Educator , National School 
Boards Journal , Phi Delta Kappa n , PTA Today , etc.). 

Dissemination of Research Products 

Based on its research, the Center may develop prototypic materials 
and processes, and conduct an active dissemination program to help 
schools use these products. Two classroom instructional processes 
resulting from Center research - Student Team Learning (STL) and 
Team-Assisted Individualization (TAI) — are being used in approximately 
10,000 schools by 30,000 teachers, thus affecting about a million students. 
In additional, more than 200,000 copies of the Quality of School Life 
scale have been distributed to school district personnel for measurement 
use. The recently developed Effective School Battery (ESB) is already in 
use by about 100 schools. 

The strategy of the Center to disseminate educational processes and 
measurement instruments that can be used in schools and classrooms is to 
coordinate with existing dissemination outlets that offer nationwide 
coverage and expertise. The Center ties its products into commercial 
publishers' marketing capabilities, the National Diffusion Network, the 
Regional Laboratories program of the Office of Educational Research and 
Improvement, and other federal and state financed dissemination outlets. 
Also, the Center coordinates a nationwic.e system of certified trainers in 
conjunction with its dissemination of Student Team Learning and Team- 
Assisted Individualization. 

1. Commercial Publication — The Center seeks commercial 
publication and marketing of its educational processes and instruments 
when appropriate. The Center's TGT program was published and marketed 
by Argus Communications, Inc.; The Quality of School Life Scale has been 
published by Riverside Press (a subsidy of Houghton Mifflin); Mastery 
Education Corporation is publishing and marketing the Team-Assisted 
Individualization (TAI) math program, and the Effective School Battery is 
published by Psychological Assessment Resources, Inc. 

2. National Diffusion Network (NDN) — the NDN consists of 
facilitators in each state who assess school district and teacher needs 
within their states and suggest the use of educational products which 
meet those needs. The products are those validated by the Office of 
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Education-Office of Educational Research and Improvement Joint 
Dissemination Review Panel as having proven education effects. 



The Center followed OERI guidelines to validate the Student Team 
Learning and Team-Assisted Individualization processes in order to put 
them into this network. This network participation facilitates the 
Center's awareness activities, teacher training, and follow-up throughout 
the nation. Dissemination of the STL process was funded through the 
NDN for five years; the dissemination of the TAI process has been funded 
for two years. We will seek validation and dissemination through the 
NDN for many of the classroom instructional processes that will be 
developed in Center work in the next five years. 

3. OERI Educational Laboratories — The Center's educational 
products are placed into the national dissemination network formed by the 
educational laboratories of OERI. Very good dissemination of research 
findings and products have occured through some laboratories, and much 
less through others. The Center works closely with the OERI effort to 
make the Lab-Center collaboration more effective. 

4. Other outlets — The Center also initiates contact with and 
coordinates dissemination activities with other appropriate federal and 
state outlets that have dissemination capability. These include 
Desegregation Assistance Centers, state department of education 
dissemination projects, NEA and AFT, and other education associations. 

The general dissemination plan described above incorporates a 
multitude of specific activities. The activities are primarily the 
responsibility of the Associate Director who collaborates with Center 
program and project directors to identify and carry out the activities 
appropriate for each research project. 

Questions/Discussions on Dissemination Issues 

A national education research center that is in the early stages of 
developing a dissemination program faces certain critical issues. This 
section of the paper, contains some personal opinions — based on 
experience and observation — about these issues. 

Should the Center have a full-time dissemination person ? Absolutely. 
The Center should have one high-echelon person whose responsibility is 
to work day-in and day-out with all Center research projects, identify 
the audience and dissemination needs of each project, set up a Center- 
wide office to respond to information requests, maintain inventory, handle 
publication of technical reports, newsletters, and special communications; 
write news releases, articles, research summaries; collaborate with other 
organizations and organizations and associations and so on. 

Some of these activities and responsibilities could be handled by 
bringing in consultants; some could be handled by working with a public 
relations firm; some could be handled by a lower echelon writer/editor 
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position; some could be handled by research personnel. But all can be 
handled by a full-time high-echelon dissemination person. 

Should the dissemination person do research on dissemination ? No. 
Different skills and mental attitudes are required in doing dissemination 
and doing research on dissemination. Communication skills (especially 
strong writing skills), a cooperative and helpful attitude, and a "small 
ego" are needed by the disseminator, who must clearly interpret and 
promote other people's work and careers for the good of the Center. 
These are not the skills required of a good researcher. 

The dissemination person will, of necessity, do research in the sense 
of documenting dissemination activities and evaluating the results, but 
this is Center-specific and project-specific. 

What methods of dissemination would be best for the Center to use? 
The quick and dirty answer to this question is — all of them. And this 
is a serious answer. It may take time to get a variety of dissemination 
methods and outlets established, but one flows into another and 
interactions abound. 

A beginning Center should think in terms of two basics — Center 
technical reports, and a Center newsletter. From these two basics, a 
full-fledged dissemination program can be built. 

Technical Report Series . The description and findings of each 
center research project should be published in detail in a technical 
report that provides the information base for dissemination. This 
report links directly to research conference presentations of the 
same information by the researcher, which may occur before or after 
the report; placement of the information m ERIC, which involves 
time for processing; and official/commercial publication of the same 
information as a journal article or book chapter, which occurs long 
after the original research is conducted. The Technical Report 
Series offers two great advantages — timely publication of the 
research, and a base of information to be disseminated by the 
Center. At the same time, the Center's production of technical 
reports is inexorably linked to dissemination through research 
conferences, the ERIC system, journal article publication and book 
chapter publication — all of which combine to get research 
information out in a credible form to the research community and 
professional educators. 

Center Newsletter . This should be the major vehicle for 
officially telling the education public what the Center's research has 
found. Newsletter is not a good term — it should be a report or 
research findings, a presentation of information that readers can 
apply or at least use to broaden their point of view. 

Here I'll parrot what everyone says — the newsletter articles should 
be clearly and concisely written, free of unnecessary jargon. But 
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remember that educators are the audience, and some jargon communicates 
clearly and directly. 

As with the Technical Report Series, the newsletter publication 
interacts with other dissemination activities. An article prepared for the 
newsletter, with little rewriting, becomes a press release to the education 
media. It becomes a summary of findings to be used by ERIC. It forms 
the base for a lengthier article written for an education association 
magazine. It becomes a summary to be submitted to the R&D Report , the 
dissemination arm of the Regional Laboratories. It becomes an expanded 
piece to be sent out direct mail to a special-interest group. 

Thus a beginning national research center, by concentrating on the 
production of a Center Report Series and a Center newsletter that 
presents research findings, will effectively work its way into a full- 
fledged dissemination program. 

Should the Center establish a dissemination "office?" Yes — an 
office that has the full capability to deal with success. Dealing with 
success means being able to respond to the consequences of effective 
dissemination — a multitude of requests for reports, publications, 
products, and Center information. 

Most universities have the capability to set up some type of 
"revolving account" to finance this type of activity. Costs of personnel, 
printing, mailing, and so on are reimbursed to the account by charging 
for publications. 

To what extent should the Center work with national associations 
and organizations? The Center should identify its natural allies (e.g., as 
the Maryland Center has identified IEL, ASHE, and others) and work 
closely with them — but care is needed. All national associations and 
organizations have their own agendas, and can sap the strength of a 
research center with too many conferences, presentations, collaborative 
meetings, and special projects. To the extent possible, the Center should 
seek to provide national associations and organizations with research 
information and let them run with it while the Center continues to 
conduct further research. To the extent possible, the Center's 
dissemination person has to help strike a balance between getting 
researchers to work with associations and other organizations while 
protecting the time of the researchers so they can accomplish their 
research goals. 

Relationships with associations and other organizations can be 
fleeting and difficult to maintain on a personal basis, as officers and 
personnel change and priorities shift. But if the Center builds a 
relationship based on providing good research information to the 
association — not a relationship built on personalities — then a more 
solid relationship will be built. 

At the same time, personal contact is important with the selected 
associations and organizations whose interests correspond directly to the 
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Center's interests. The Maryland Center would want to pursue personal 
contact and interaction with the Higher Education Clearinghouse, for 
example, and with the Chronicle of Higher Education writers and editors. 

What are the most important factors in effective dissemination of 
educational research? The most important factor is well-conducted 
research that produces strong research findings that are useful to 
someone. Given this factor, the disseminator's life is an easy one. 

The nature of education research, however, is that is usually 
produces a small increment in knowledge in a small area and, although 
well done, still has some flaws in the methodology. Given this, the most 
important factor in dissemination becomes the ability to interpret the 
immediate and future impact of the research and to convey this 
interpretation in a straight-forward but exciting way. Thus interpretation 
skills and writing skills are two vital components of effective 
dissemination. 

I would consider the third most important factor to be the 
identification and use of all possible dissemination outlets. A multitude 
of people and organizations exist whose primary purpose is to provide 
other people and their constituencies with full information about 
education — the popular press, national magazines, the education media, 
the associations, advocacy groups, the ERIC system, regional laboratories, 
and many others. Effective dissemination consists of getting a clear, 
important, well-written message to these outlets, so they can provide it 
to their constituencies. 

Will dissemination of Center research improve higher education? No, 
not directly. Here I make the real distinction between dissemination of 
information and educational improvement. Simply disseminating 
information does not produce educational improvement that can be 
documented in a coherent way — improvement may happen, but tne links 
between dissemination and improvement are vague and slippery. But 
effective dissemination is at least a first step toward improvement. 

Documentation of dissemination activities will provide a base for 
assuming higher education improvement. The Center can show that its 
research findings reach the people who can apply them to achieve 
improvement. It can survey the people who receive its findings and 
document their use of the research. It can then assume that the use 
resulted in some kind of improvement of higher education. Proof of the 
bottom line — that management of higher education institutions improved 
or that American college students improved their achievement as a direct 
result of the Center's work — may never be satisfactorily documented. 

However, the Hopkins Center experience with trying to improve 
schools tells us that product development can bring a Center closer to 
achieving educational improvement. Research findings that indicate what 
educators should do are important, but insufficient. The findings need to 
be extended into research products that the practitioners can apply, step- 
by-step, to achieve improvement. It remains to be seen whether this is 
as trues in higher education as we've found it to be in K-12 education. 
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MESHING THE GEARS; 
REPORTING ON HIGHER EDUCATION IN AMERICA 



Think of the Center as a machine. Somewhere at the heart of the 
machine is a set of gears — gears of large diameter which turn slowly 
(no doubt exceedingly slowly in the estimation of some), gears of small 
diameter that turn quickly. Also think of the audience the Center wants 
to reach as a much larger machine, with many gear boxes. The secret to 
successful dissemination of the news of Center activities is to mesh your 
small gears with your audience's small gears and your large gears with 
your audience's large gears. To put it another way, you have to tailor 
your "product" to your "consumers." More on that later. First, a brief 
look at what education news means to Americans and a walk through the 
industrial park we call the "media." 

Education News in America 

There's no question that education news is important to people. 
Both higher education and elementary-secondary education are vast 
enterprises that employ huge numbers of people, account for a significant 
proportion of the gross national product and, most important, embrace an 
experience that EVERYONE goes through at one time or another. (Thus, 
there is no shortage of experts in education. In newsrooms, editors will 
defer to science and political writers, but they tend to exert themselves 
forcefully on education matters. After all, THEY went to school and 
college, and their children are in school or college. Many, perhaps the 
majority, of story ideas are generated by editors and reporters as a result 
of something that is happening to their own children and those of their 
neighbors.) 

A number of polls and readership surveys conducted by news 
organizations have shown that people follow education news. The 
Baltimore Sun's privately conducted readership surveys indicate a 
conristently high interest in education. The Education Writers 
Association (EWA) last conducted a national opinion survey in 1977. To 
summarize its 10 major findings: 

o People want to read, watch and hear news about education, 
higher and "lower." 

o People get most of their news from newspapers and TV. 

o They read newspapers and watch TV rather than listen to the 
radio for information about their schools and colleges. 

o To learn about general education issues, people prefer 
newspapers to TV and radio. 

o The nation by a slight majority believes the media tell people 
what they want to know about their schools. 
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o People believe by a slight majority that the media tell them 
what they want to know about general issues in education. 

o In regard to the coverage of elementary-secondary news, people 
say that "discipline," "spending" and "subjects being 
taught" are the topics they want discussed by the media. 

o The three top general education issues people want covered 
more by the media are "federal policies and programs," "trends 
in curriculum" and "national testing of students." 

o Americans believe overwhelmingly that the media present 
education news so they can understand it. 

o Of those people who believe that the media do not present 
education news in an understandable way, the greatest number 
list "vague statements" as the main reason. 

This EWA survey was conducted nearly a decade ago, but I believe 
the findings are generally true today. (Annual Gallup polls of education 
substantiate my belief.) 

If there has been any change in the decade, it is in the direction of 
MORE interest in education. Certainly this "observation is upheld by 
membership surveys conducted in 1985 by the EWA. We found that 
members have been on the beat longer, that they spend more of their 
time reporting education, and, most important, that their news 
organizations place more emphasis on education, giving them more time, 
for example, to do major "projects." 

No doubt much of this was brought about by the Great Reform 
Movement. The 1983 federally sponsored "National at Risk" report, 
followed as it was by a spate of generally critical reports by commissions, 
task forces and committees, served to spur education interest as state 
after state, local school board after local school board, began "reform" 
efforts. It's important to note that most of that reform has occurred at 
the state and local levels, providing hundreds, probably thousands, of 
good stories for newspaper, radio and TV reporters, not to mention 
writers for the Chronicle of Higher Education , Change , and 
specialized/academic journals. EWA members have followed these 
developments through its publications and through the bimonthly "clip 
sheets" of the Educational Excellence Network, now at Teachers College 
but founded at Vanderbilt by (among others) Chester Finn and Diane 
Ravitch to monitor the so-called "excellence" movement. 

Meanwhile, in the mid-1980s, the movement spread — graduated, if 
you will — to higher education. James L. Fisher, former president of 
Towson State University and of the Council for the Advancement and 
Support of Education, traced this development on January 9 in the 
Baltimore Evening Sun . It began, perhaps, with Terrel Bell's "Investment 
in Learning," which urged a national debate on issues in higher education. 
Shortly afterward the National Endowment for the Humanities issued "To 
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Reclaim a Legacy," which concluded that "too many students are 
graduating from college lacking even the most rudimentary knowledge of 
history, literature, art and philosophical foundations of their national and 
civilization." 

Next came a report from the American Association of Colleges which 
declared that a "misguided marketplace philosophy has led to an 
indisputable deterioration of the curriculum." Then the first Carnegie 
report, "College and the Undergraduate Experience," concluded that the 
general state of curriculum was "lamentable" and that commercialism in 
admissions was "scandalous." This was followed by another Carnegie 
report with essentially the same filings. The culmination, perhaps, was 
the 1986 report of the Carnegie Foundation for the Advancement of 
Teaching, calling for a "complete overhaul" of undergraduate education. 

THE MEDIA 

Much of the foregoing discussion has been rather up-beat. Now for 
the down side. Most news organizations were — and are — ill-prepared 
for the shift to higher education. A great irony is that despite the 
reform movement, despite the general increase in education news, despite 
the fact that most major newspapers (including my own) devote more 
resources and news space to education than ever before, education 
remains a lowly beat. 

Roger Yarrington, who left the job of associate dean of the College 
of Journalism, University of Maryland in 1975 to return to serve as 
communication officer for a national religious body, conducted a survey of 
the education press in 1984. His major conclusions were: 

o Most education writers concentrate on secondary and 
elementary education. 

o Many are assigned to additional beats. 

o Few focus on higher education exclusively. 

o And the leading ones tend to work for papers with policy 
commitments to provide consistent coverage of education. 

My own observations as a 21-year member of the 47-year-old EWA 
confirm Yarrington's. I could add that the education beat on a paper is 
often assigned to the beginning reporter, alcng with obituaries and police 
reporting. Many editors define the beat as school board watching (and 
an occasional bright feature on a "unique" local program). 

But education writing can be a reporter's plumb. As Saundra Keyes, 
former education writer for the Nashville Tennessean and Louisville 
Courier-Journal (now deputy managing editor of the Orlando Sentinel), 
observes, education has built-in intsrest for readers because it involves 
two things they value intensely: their children and their money. 
Education accounts for nearly half of state and local government 
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expenditures in most states. Moreover, the beat cuts across more lines 
than any other in a news organization: politics, religion, business, the 
courts, sports, race relations, social issues and more. 

All of these observations are as applicable to higher education as 
they are to elementary-secondary. But there simply aren't many higher 
education reporters out there. If we exclude what I call the 
specialized/academic press ( Chronicle , Change , etc.), there are no more 
than a dozen full-time higher education reporters in the U.S. They are 
at a few newspapers with heavy interest in education ( Baltimore Sun , 
Milwaukee Journal , Detroit News ). The Boston Globe assigns a full-time 
higher education reporter, but no wonder. Colleges and universities are 
big business in Boston. 

Yarrington found 12 higher education reporters among the 1,730 
daily newspapers in the U.S., 48 of them with circulations over 200,000. 
Looking through the Editors and Publishers Yearbook , the "bible" of the 
industry, he found only 240 men and women listed as "education writers" 
or "reporters." 

The EWA membership is at an all-time high of 523. Of those, about 
three-fourths are education writers for newspapers. (Most of the rest 
are public information people for colleges and universities.) Of course, 
the EWA doesn't include in its membership all education writers. Let us 
assume that there are 600 among the 1,730 dailies, and that almost all of 
the remaining papers have someone who covers education on a part-time 
basis. I know of no TV station with the full-time education specialist. 
We have had one TV member (he is no longer active) in the entire 
history of EWA. And we have had one radio .2porter, John Merrow (still 
a member, but now of the MacNeil-Lehrer News Hour on PBS). 

Yarrington (1984) reported: 

"In the smaller papers, from 100,000 circulation on down, you'll 
commonly find the person listed as education writer listed also 
as society editor, as the writer responsible for real estate, 
farm news, books or some other department. Some education 
writers also cover gardens, outdoors, films, sports, autos and 
amusements. Two of the more promising combinations I saw 
were education writers who also covered business and wrote 
editorials. Clearly education, employment and economic stores 
are closely related. All of these data suggest that education 
writers often are in that assignment for a short time, on their 
way to other responsibilities and covering education in addition 
to handling other stories. Even the ones who emerge from my 
inquiry as higher education specialists usually indicate they 
spend mere time on secondary and elementary than on higher 
education." 

Yarrington's survey results are consistent with my experience. Two 
years ago I was the coordinating editor of a two-year project in Indiana 
sponsored by the Indiana-based Lilly Endowment. Lilly was concerned 
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about the state's poor record in economic development and its dreadful 
record in attracting and graduating black students. Four journalists in 
1985 and four in 1986 were given generous fellowships to exami. e one 
aspect of higher education — it was hoped but not required that the 
topic would fall in one of the above subject areas — and to write a 
series for their home papers (or produce a series for their home TV or 
radio stations). We had 10 applicants the first year and seven the 
second. Even though word had spread about the first-yeai. experience, 
even though two of the first-year fellows had won well-publicized awards 
for their efforts, we had clearly scraped the bottom of the barrel by the 
second year. Two of the papers participating in 1985, the Gary Tribune 
and the Indianapolis Star , were repeats in 1986. And despite our efforts 
to publicize the program, the papers in Bloomingtcn and Lafayette, home 
of Indiana University and Purdue University, respectively, did not apply. 
(The public radio station at Indiana University did participate the second 
year ) 

>.e have to assume, then, that the vast majority of U.S. education 
reporters, and of reporters assigned, however smporarily, to education, 
spend little time on higher education. There is evidence, however, that 
that is changing and that more time and effort are being expanded on 
higher education. The challenge for the Center is to take advantage of 
that historical moment. 

What do education reporters look for, and how do they operate? 
What is the "culture" of the ne*'sroom? From the above, it must be 
obvious that education reporters for newspapers are in heavy competition 
with other beats if they specialize exclusively in education. Those 
reporters for whom education is a part of their responsibilities have even 
less time to devote to school and college matters. 

TV reporters are almost always generalists. In large metropolitan 
areas, they arrive at "scenes" — a school "riot" has broken out, the 
morning newspaper has published a good feature, an anti-desegregation 
group has called the assignment editor to inform him or her that a 
demonstration is planned for 3 p.m., word is out that the evening school 
board meeting will feature a hot discussion on sex education. Cften, the 
reporter has just reported on an auto wreck or mayoral press conference 
and has little Idea of the background of whatever he or she has been 
assigned to report. (Newspaper reporters spend much of their time 
briefing TV reporters, who are notoriously ignorant of education trends 
and issues.) 

Radio reporters of any stripe are a dying breed. "Once the 
backbone of electronic journalism and the first source of live reporting, 
radio news is on the skids," The N(*w York Times reported recently. 
Following the deregulation of radio — stations are no longer required to 
devote specific time periods to public affairs programming, and the 
Federal Communications Commission no longer keeps records of , dio 
news employees — many stations have cut down on news staffs. The 
number of all-news stations h&a stabilized and declined in some cities 
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(from three to one in Washington, D.C.)i and in most major cities the 
number of radio reporters "on the street" has declined to a precious few. 

So we are talking largely of newspaper reporters and editors. Most 
of - them, as they look at education news from national sources, are 
looking for application. There are a few exceptions to this. "National" 
newspapers like the Wall Street Journal, the Christian Science Monitor 
and USA Today are looking for national stores. But reporters in 
Columbus, Ind.; Bend, Ore.; Las Vegas, Nev.; and, yes, Portland, Ore. and 
Baltimore, Md, are under constant pressure to "localize" stories. 

The reason for this is simple. Again with a few exceptions, almost 
all education reporters are assigned to local desks. Their bosses are city 
editors. Their assignments include covering the loca' school board 
meetings, an excruciating experience which leads many talented journalists 
to get out of education reporting as soon as possible. They look for 
stories that can be "plugged in" locally. They compete with other beats 
on their papers for increasingly sparse space in a diminishing "news hole." 

Let me give an example. Four years ago at the EWA national 
seminar in San Francisco, we arranged a session on the forthcoming 
"Nation at Risk" report. It was April, and though the national 
commission's report wouldn't be out until mid-May, two commission 
members agreed to appear on a panel on Friday afternoon. One of them, 
for reasons known only to him, spilled virtually all of the beans. Our 
members took copious notes, and I took care to see what kind of 
coverage the session produced. 

Almost without exception, papers that carried the story carried it 
only if local comment could be obtained. Even though they had sent a 
reporter to San Francisco, editors were not interested in a story about a 
reported national decline in educational quality if it could not be 
localized, with comment from the superintendent or school board 
president. Such was not the case six weeks later, when the report came 
from the White House, when it was accompanied by remarks from the 
President and when it contained phrases such as the "rising tide of 
mediocrity." Associated Press stories appeared on Page 1 of newspapers 
across the nation. 

Education reporters want: 

o Stories that can be localized 

o Stories that allow comparisons of their towns, cities and states 
with other towns, cities and states. We are a nation of 
comparers. We dote on Top 10s and Bottom 10s. Our sports 
pages are full of such statistics. Editors love them. I'm 
convinced ihat the tremendous interest in SAT scores, an 
interest that has lasted almost two generations, stems from this 
deep-seated desire to compare "us" with x. 
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o Stories that can be made into graphics. This is true of 
newspapers and TV. It's an era of editorial graphics, and no 
self-respecting newspaper is without them. 

o Man bites dog. Yes, it's still true. People don't want to read 
or hear about business as usual in schools and colleges. But 
they will read about the unusual, the explosion of a myth, the 
deviation from common thinking. 

o Reports. Often, reporters aren't very critical in evaluating the 
contents of a report, but the report automatically covers much 
of the ground the writer has to cover in preparing a news 
story — who, what, why, where and when. Just the fact that 
it IS a report makes it authoritative. How many leads have 
you read in how many newspapers and journals that end, 
"according to a new report"? The current reform movements in 
higher and elementary-secondary education have been fueled by 
reports. 

o Research, particularly if it fits the above criteria. Again, 
reporters do not ask enough questions about research; the/ 
don't know how to distinguish the good stuff from the junk. 
By and large, they don't know ho* to go about obtaining good 
research. And only the more sophisticated go beyond the 
research they see to ask a question posed by Ward Mason in 
his paper for the National Center for Postsecondary Governance 
and Finance: What is the relationship between research and 
practice? 

An EWA board member, Larry J. Hayes, editorial page editor of the 
Fort Wayne Journal Gazette , believes the "gap between theory and 
practice" is the biggest story in education but also one that has gone 
largely unreported (Bulletin of American Society of Newspaper Editors, 
1986). Speaking at the 1985 meeting of the American Society of 
Newspaper Editors, Hayes advised journalists to learn more about 
research, to learn to distinguish good research from bad, and to develop 
ihelr own network of experts. 

Meshing Gears 

It seems to me that the Center, if it chooses to, can make a huge 
impact if it learns to mesh its gears properly (to return to my original 
metaphor). This is because governance and finance are two particularly 
ripe topics. (Maryland, for example, will examine its system of higher 
education governance as the result of a report leaked to the press in 
early January.) Several of the Center's initial projects are of wide 
interest. Take John Lee's investigation of "differential pricing." It fits 
in nicely with the holiday report of the Congressional Joint Economic 
Committee, and Lee's findings ought to be of vital concern to the 
millions of American paying college tuition. 
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I agree that dissemination should be a major part of the Center's 
activities. From the moment a research project is but a gleam in your 
eye, you ought to be thinking about spreading the word. Different 
audiences will have different needs. There are at least four main ones, 
three of which I know something about. 

1. The academic audience, reached through journals and 
specialized publications. 

2. The higher education journals such as Chronicle and Change . 

3. The national popular media such as The New York Times , 
Newsweek and the networks; 

4. The daily papers around the country whose editors and 
reporters will ask one major question about your findings: 
What do they mean for us? 

The Center should get to know some of the major actors in these 
media. Find out how Time magazine and the Wall Street Journal and NBC 
operate, who is responsible for education at each of these places; how 
they like to be informed, how you can help them do their job. These are 
obviously the Center's big gears, and there is nothing to replace personal 
contact. 

Some of the audiences overlap; a professor at the University of 
Wisconsin-Madison might read the Wisconsin State Journal , The New York 
Times , the Chroni cle, the Bulletin of the AAUP ; listen to "All Things 
Considered" and watch Dan Rather and MacNeil-Lehrer. It would be 
difficult for the Center to have contacted all of these sources, but it 
could have thought beforehand which media outlets it wanted to contact 
and in what way to contact them. In a sense, the Center would have 
controlled its own news. 

Two other pieces of advice: Do not fear controversy, and do not 
hesitate in the right circumstances to lower your academic brows. The 
American press thrives on controversy. A reporter's lead, as I've said 
above, must of necessity distill and reduce to the lowest common 
denominator. Take advantage of it when the time is right. "A National 
At Risk" did; it was written deliberately with phrases and metaphors 
("rising tide of mediocrity") designed to attract reporters and editors. 
The report launched a long-overdue reform movement. When Russell 
Edgerton complains, as he did in the November-December 1986 issue of 
Change , that the Sunday papers had "rifled through (the Boyer report) 
and highlighted all the negatives," I say balderdash. The report WAS 
largely negative. The press wae doing its job. Higher education is due 
for a major overhaul. 

In this respect, the "Consensus Conference Model" described by R.G. 
Havelock appears to be very useful. Not only does it open the decision- 
making process to people (including consumers) other than educators and 
researchers; it also approaches the media realistically. It recognizes that 
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sometimes you want to "make a big bang." Indeed, it accommodates the 
making of a big bang. And it recognizes "good clear writing." 

The importance of good writing is my last point. Higher education 
is so clogged with nonsense and gobbledygook, 25-cent words where 10- 
cent words would do, the passive voice, the utilization of words like 
utilization! Putting out written material in clean, straightforward English 
would not be a low-brow exercise; it would be a great service to 
humankind. 
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THE CONSENSUS DEVELOPMENT CONFERENCES OF THE 
NATIONAL INSTITUTES OP HEALTH AS A MODEL 
FOR EDUCATION AND OTHER FIELDS 



In 1977 the National Institutes of Health initiated an innovative 
program to increase expert consensus on important medical problems. 
Identified from its earliest stages as "The Consensus Development 
Program" this effort was manifested princip m \y in a series of carefully 
organized conferences which followed a distinctive pattern. The purpose 
of this paper is to describe that pattern in some detail, to discuss its 
costs, to consider its strengths and weaknesses, and to derive implications 
for fields other than medicine. The Consensus Development Conference 
has always had the avowed purpose of achieving an expert "consensus" 
where a significant amount of research was available. Thus it involves 
both knowledge synthesis and some arbitration of differences among 
leading researchers. However, from the beginning these meetings have 
also been seen as a means of building bridges between research and 
practice and more specifically between clinical research specialists and 
primary care physicians. Other purposes included technology assessment, 
diffusion of medical knowledge to the general public and to interested lay 
communities, and (lastly) articulation of new research agendas and 
stimulation of needed research. 

To date, there have been about 60 such conferences, and they have 
covered a very wide range of medical topics. Some conferences focus on 
a particular technology either for diagnosis (e.g. CT scanning) or 
treatment (e.g. electroshock therapy). Some focus on treatment modalities 
for a given disease (e.g. adjuvant chemotherapy for breast cancer). 
Others focus on health consequences of certain behaviors (e.g. 
consequences of obesity and dietary habits, habitual use of smokeless 
tobacco). Topics are determined by representatives of the various 
bureaus, institutes, and divisions within NIH and sometimes proposals from 
the National Institute of Mental Health are also entertained. Since 
conferences are partially funded by each of these agencies, topics are 
designated in a loose kind of rotation, roughly proportional to the size of 
the unit in question. Thus, there are more conferences convened on 
cancer-related topics than any other because the National Cancer 
Institute is the largest unit of the NIH. 

Over-all management of all the conferences is in the hands of p 
small office under the NIH Director, the Office of Medical Applications of 
Research (OMAR). This office is responsible for maintaining a consistent 
CDC process, arranging publicity before and after, supporting the work of 
the panel, and evaluating both process and impact. Much of this work is 
carried on through contractors. The CDC process has been described and 
analyzed in a number of brochures, reports and other publications (e.g. 
see Perry and Kalberer, 1980; OMAR, 1983; OMAR, undated; Jacoby, 1983). 
The present paper is based on these sources augmented by several months 
experience as director of a support contract to OMAR in 1985 and 1986. 
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There also have been two formal evaluation studies. The first, by 
the University of Michigan, covered the early years of the program and 
focused on the process itself. The second, by the Rand Corporation, 
studied the impact of selected conferences on medical knowledge and 
practice. OMAR has also conducted a few surveys in connection with 
specific conferences and routinely asks panelists to provide feedback on 
the process on a standard form. Unfortunately, very little from these 
evaluative efforts has been published or is publicly available as of this 
writing. However, an examination of unpublished material from these 
sources suggests that the CDC process has become an accepted model in 
the medical community, is very well regarded by panelists and by 
conference participants, and has some impact at least in the awareness 
and knowledge of large numbers of medical practitioners. 

Perhaps the most important point to emphasize here is that the CDC 
represents a coherent and distinct social innovation which has achieved 
acceptance within NIH and within the medical community during a time of 
fiscal stringency, having overcome widespread initial skepticism, and 
competing successfully in a veritable sea of medical meetings and 
publications. It has also been emulated as a model for medical technology 
assessment in both Sweden and the United Kingdom although the Swedish 
and UK variants have some significant differences, putting more emphasis 
on financial, ethical and other non-technical aspects. 

This paper will be limited to a consideration of the CDC as an 
innovation in the social processing of knowledge with implications for 
fields other than medicine. 

The Process as a Whole 

The focus of all CDC effort is the consensus statement a document 
of roughly 12-20 typed pages, carefully crafted to respond to a set of 
questions previously posed in the conference planning process. The 
statement represents the collective judgement of a panel of experts with 
somewhat diverse credentials. NIH goes to some lengths to make the 
process appear legitimate, rational, and informed. They do this through 
careful selection of panelists, through assembling all leading authorities 
on the subject as providers of expert testimony, and through creating an 
open forum so that all interested parties are given an opportunity to 
give input. NIH is also at pains to point out that it is merely the 
convener of the conference, not the promulgator of the statement which 
thereb/ does not represent an official NIH or government position. 

The panel sits on stage before the assembled conferees for one and 
one half days somewhat like a panel of judges, receiving testimony and 
interrogating those who provide it. At noon of the second day the panel 
retires for an intensive writing session which may last up to 16 
continuous hours. Each panel is charged with producing a consensus 
statement responsive to all the pre-set questions by 9:00 a.m. of the third 
day of the conference. Each of the 60 panels to date has managed to 
meet this deadline. The text, stamped "draft" is then distributed to all 
those present and a further round of questions and editing suggestions is 
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entertained. The panel then retires for a final brief review and editing 
session. This edited version is then distributed to the press at a news 
conference convened for 12:00 noon. 



Following the conference the statement is printed as a pamphlet and 
mailed to p larger audience of interested persons, primarily physicians. It 
is also published in at least one prominent medical journal, usually the 
Journal of the American Medical Association. From time to time other 
forms of dissemination are also attempted. 

The Consensus Conference process divides clearly into three phases: 
the pre-conference preparations, the conference itself, and the follow-up. 
Substantial effort is expended on each phase, and adequate performance 
of each phase is essential to the over-all result. Each of these major 
phases also subdivides into a number of task elements. Some of these 
elements have been refined by NIH to a point where they are performed 
smoothly and efficiently time after time in almost precisely the same way. 
Some other elements are more variable in execution, depending on the 
topic selected and the mix of persons selected to make a contribution. A 
few elements, some important, continue to cause problems and are the 
subject of continuing concern within OMAR and of some criticism from 
outsiders. 

Elements of The Process 

The Prs-conference Phase . The success of the consensus conference 
process at NIH can be largely credited to elaborate and extensive 
planning, both for the process in general, and for each specific meeting. 
The pre-conference phase can be divided into roughly five task areas: 1) 
the identification of the topic, 2) the convening of a planning committee, 
3) the assembly of the knowledge base, 4) advance publicity, and 5) 
logistics. 

1. Identification of topic . Arguably the most important CDC 
task is the initial selection of the topic. Strangely, however, this is 
an area which has little formal delineation. There are some rather 
broad formal guidelines, for example, conferences "...may examine 
either emerging or established technologies," and "...are particularly 
useful for providing guidance when a controversy exists in differing 
therapeutic or diagnostic options and the issue is of public as well 
as professional interest" (OMAR, 1983, p. 2). Such language does 
not exclude very much from consideration. 

On the other hand, there are a number of unwritten rules. 
The most important of these is that each of the Institutes takes its 
turn at proposing topics, very rough) y in proportion to the fraction 
of the over-all NIH budget as previously noted. Some other 
unwritten rules usually but not always followed are: a) not too new; 
b) not too broad; and c) not too controversial. 
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a* Not too new * Most topics concern technologies that 
have already been in wide clinical use throughout the United 
States* Thus there has occasionally been criticism that 
conferences deal mostly with established rather than new 
technologies and are following rather than leading the state-of- 
the-art. The counterargument is that truly new technologies 
will not have an adequate data base behind them to be 
objectively evaluated* Further, there are other channels for 
introducing such new technologies and this is not the primary 
purpose of the CDC* 

k« Not too broad * NIH tries to select topics which are 
of direct and practical interest primarily to physicians* As 
stated in the Guidelines pamphlet, "the primary focus of the 
CDC is the technology's clinical applications" (OMAR, 1983, p. 
2). Frequently this results in tho choice of topics which are 
remote from the awareness and understanding of the lay public, 
couched in terms that are comprehensible only to a medical 
audience. On those occasions when broad topics are selected 
(e.g. twice ir* recent years conferences have focused on the 
treatment of pain) the panel has had some difficulty coming to 
terms with the topic and delivering a concise statement which 
has value either to physicians or to a larger audience. 

c « Not too controversial . CDC's are unlikely to deal with 
raging medical controversies, primarily because consensus in 
such arras would not be possible. The "consensus' 1 that is 
reflected in panel deliberations and their statements will 
typically be a consensus that has already emerged at least 
among experts who have studied the problem closely. Thus, 
the statement may seem redundant with what is already k.iown 
to many in the field and may be only a reflection or 
resummarization of knowledge already widely shared and 
accepted* 

There is no formal list of potential topics and OMAR 
makes no formal periodic solicitation of suggestions from the 
Bureaus, Institutes, and Divisions of NIH (usually referred to 
internally as "the BID's"), any one of which is entitled to 
submit a topic at any time. Other conditions that inevitably 
restrict selection are the OMAR budget and staff support 
capabilities which allow only for between six and eight 
conferences per year currently and the lead time necessary for 
a conference which is typically about one year, but can be as 
short as six months. 

2. The planning committee * The first formal act of developing 
a CDC is the convening of a special planning committee for that 
particular conference. The membership of the committee is 
dominated by NIH staff but a few outsiders are also invited 
including a person who has already tentatively agreed to serve as 
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panel chairperson. The prospective chair works closely with the NIH 
conference coordinator who will also be a substantive expert on the 
topic and in many cases the chief stimulator of the topic's selection. 
Other members of the planning group include an OMAR staff member 
and another administrative official from the BID which has proposed 
the topic. On occasion topics will be co-sponsored by two or more 
BID's in which case the planning committee will be expanded to 
reflect such co-sponsorship. 

The planning committee meets formally for one day, 
supplemented by correspondence and informal meetings before and 
after this event. The planning meeting has a crowded agenda which 
includes the following tasks: (a) designation of the panel chair; (b) 
nomination of panelists; (c) identification of conference presenters; 
(d) clarification of the topic and preliminary definition of the 
knowledge base; and (e) determination of the questions to be 
addressed. 

Usually about 12 panelists are named, two or more MD 
practitioners, one or two clinical researchers, one or two 
methodologists, a lay person who may be an active member of a 
disease society or a consumer organization, and, depending on the 
topic, a legal expert and/or a medical ethicist even though legal and 
ethical issues are not supposed to be addressed in any depth. A 
most difficult aspect of the planning committee's work is selecting 
panelists who are highly competent to judge the state-of-the-art but 
are not at the same time known to have publicly stated positions on 
the topic. Occasionally, panels and panel chairs have been criticized 
on grounds of bias by stakeholders with opposite views to those 
expressed in the "consensus." However, it is manifestly difficult for 
a panelist to have been associated in the literature with a topic 
over a number of years or in a number of publications without 
having come to any stated conclusions. The planning committee 
therefore has to make a fine distinction between someone who is 
generally respected as a medical authority in a general field or a 
related field and someone who is an expert on the particular topic 
addressed. 

Identification of appropriate conference speakers is a somewhat 
easier task. These will either be experts in the field, or prominent 
stakeholders on the topic (e.g. industry and disease organization 
representatives). The medical research community is very large but 
also very well inter-connected with the scientific norm of refereed 
publication of all findings strictly adhered to. Added to this is the 
central role of the NIH in funding research on nearly every 
important medical problem. Therefore, when a planning group of 
five or six persons familiar with a particular issue come together, 
they are almost certainly able to 'identify the loading researchers 
and advocates for various positions. 
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An equally important and more controversial role of the 
planning meeting is the formulation of questions to be put to the 
panel. There are usually about six questions. The first question 
typically asks the panel to define the nature of the phenomenon. 
Another typical question asks about the scope of the public health 
problem or challenge deluding data on incidence and prevalence in 
the case o2 disease, trauma, or health-risk behaviors. There will 
always be a question about risks and contra-indications for any 
technology or procedure which is under investigation. Other 
questions will likely deal with causes, populations at risk, diagnosis, 
prevention, and treatment. The final question invariably asks what 
further research is needed on the topic. 

Because the planning committee is meeting only for one day 
with a heavy agenda, the opportunity to explore nuances of the 
topic, to consider its true scope, and the depth of literature that 
might be available to support panel deliberations is very limited. 
Nevertheless, an attempt is made to scope the topic somewhat 
informally and major omissions are rare. 

Once a topic has surfaced and a planning committee 
established, the process is most likely to proceed to a conference 
and the conference in all but one instance has resulted in a 
"consensus" made explicit in the consensus statement. Thus there is 
no precedent for outright rejection of a topic by a planning 
committee. On the other hand, planning committees often will work 
over a topic, broadening or narrowing the scope or changing 
emphasis. Furthermore?, results of the planning effort are subjected 
to a final review and sign-off by the Director of NIH and changes 
are occasionally made at that level also. 

3. The assembly of the knowledge base . Another important 
task of the preparatory phase is the retrieval and packaging of the 
knowledge base in a form which can be absorbed by the panel and 
presented to the conference. Three types of activities in this 
category should be noted: a) preparation of the book of abstracts, 
b) the NLM literature search, c) and the occasional use of experts 
in decision analysis and meta-analysis. 

a. The abstract book . Well in advance of the conference 
each major speaker is asked to prepare an extended abstract of 
their presentation including key data with illustrations and 
tables* These "abstracts" are, in effect, mini-articles. They 
receive minor editing l>y OMAR but are not otherwise refereed. 
There may be as many as 30 such pieces which are assembled 
in a book which might run to 100 pages. The book also 
contains an introduction to the topic, the names of the 
panelists, th<° agenda and a statement about the CDC process. 
The abstract book is circulated to panelists in advance of the 
meeting and distributed to all conference attendees at 
registration. Although it is attractively covered and is costly 
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to prepare, this abstract book receives no further dissemination 
and is never published as such, with one exception which will 
be noted later. The importance of the abstract book cannot be 
overestimated. It tends to become the bible of the conference 
and the chief reference tool for the panelists. The abstract 
book invariably gets high praise from panelists. 

b. The NLM literature search . A representative of the 
National Library of Medicine sits in on each planning 
committee meeting and is charged with the responsibility of 
making a comprehensive search of the medical literature. 
Copies of a computer print-out including author abstracts are 
mailed to each panelist along with the abstract book. This 
compilation can sometimes run to hundreds of items and is not 
generally viewed by panelists as nearly as helpful as the 
abstract book as svch. NLM does no interpretation or 
synthesis and only a minimal amount of sorting and weeding of 
items. Experience suggests that library state-of-the-art even 
in medicine still requires a considerable amount of human 
intervention. 

c. Decision analysis and meta-analysis projects . In 
recent years OMAR has made a few attempts to bring experts 
on decision analysis and/or knowledge synthesis into the 
process. Usually, such experts have worked more-or-less in 
parallel with the panel, doing their own data collection and 
analysis, then presenting their conclusions or decision-tree 
frameworks either to the conference as a whole or to the panel 
in chambers. Sometimes these efforts are rated as helpful by 
panelists but so far none appears to have had a decisive role 
in panel deliberations or the framing of the statement. This 
may be partly because OMAR has been extremely cautious in 
offering such services to panels, not wanting to disturb what 
they see as a basically effective process. 

4. Advance publicity . OMAR invest weeks of effort and 
many hundreds of dollars in advance publicity for all conferences. 
An effort is made to acquire mailing lists of all medical specialists 
who might have an interest in the topic. Glossy colored brochures 
are then mailed to a general list of thousands in addition to these 
special lists. Total distribution of advance flyers is between 17,000 
and 20,000. There are also advertisements in prominent journals and 
considerable local promotion on bulletin boards, etc. The avowed 
primary purpose of all this activity is to give every concerned 
individual an opportunity to contribute to the process. It is not 
known how much this actually contributes to widespread 
participation. Attendance at conferences generally ranges between 
100 and 300, depending on the breadth and perceived importance of 
the topic. Roughly half of those who attend apart from invited 
speakers are from the local 3-state area and a large proportion of 
these are affiliated with NIH or other federal health-related 
agencies. Physicians who attend earn continuing education credit 
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from the NIH Clinical Center which is an accredited medical 
education facility. 



5, Logistics . All conferences to date have been held on NIH 
premises, usually in their major auditorium. Expenses but no fees 
are paid to panelists and speakers and special accommodations are 
arranged for them at a nearby hotel. All ^ther participants are 
strictly on their own regarding arrangements and expenses. The 
OMAR support contractor provides a special facility where panelists 
hold their closed deliberations and are given clerical support and 
meals during the extended writing session. Although there are 
numerous advantages and some disadvantages of having the same site 
time after time, cost and convenience to NIL are probably the prime 
considerations and no other policy has ever been seriously 
considered. 

The Conference Phase . Like the pre-conference phase, the conference 
itself runs on a smooth, highly structured, and tight schedule which is 
invariant and has been since nearly the beginning of the program in the 
1970's. There are eight major component? which deserve mention: (1) 
initial orientation of the panel, (2) expert presentations, (3) stakeholder 
presentations, ^4) questions p~<d floor participation, (5) the drafting 
session, (6) the public draft review, (7) the final closed review, and (8) 
the press conference. 

1« Orientation and charge to the panel . On the evening 
before the first session the panel meets for dinner and to discuss 
their duties. For most this is also a get-acquainted session. There 
is a lead-off presentation by the OMAR director in which the 
process is reviewed and the duties of the panel spelled out. This 
presentation is invariant and is backed up by descriptive pamphlets 
on OMAR and on the CDC process. Panelists have already been 
briefed by mail but even with repetition misunderstandings msy 
persist. The panel chair then takes over the meeting. Sometimes 
the chair will entertain a brief general discussion of the topic. 
More commonly the chair will say something about his or her 
approach to the leadership task, set gjals and sometimes assign 
writing* subgroups for different questior.i. 

2. Invited expert speakers . The first full day is taken up 
with presentations by expert speakers. Presentations are made in 
clusters of three or four each, followed by a question and answer 
period on that cluster. T" ere is some attempt to group clusters in 
the order of the questions for which they are most relevant. For 
example, epidemiological studies showi tg the exter.l of the problem 
tend to come early, diagnostic procedures somewhat later, studies of 
treatment effects later still, and ethical, legal, and sociological 
issues last if at all. Some presentations will represent reviews or 
summaries over a number of researchers or a line of research done 
by an investigative team. Yet, there will also be single study 
pres.entations if these are thought to be of sufficient weight to 
merit such treatment. As noted earlier, there are sometimes special 
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presentations on the decision process that might be followed to 
reach a consensus on one or more aspects of the problem, and in at 
least one instance there has been a commissioned meta-analytic 
study, reviewing a large number of quantitative research reports 
from different authors within a common analytic framework (in that 
instance, adjuvant chemotherapy for breast cancer). 

At the close of the afternoon session the panelists retire for a 
private dinner, discussion, and work session. Usually some 
preliminary drafts of the first question are made at this time but 
there is no charge from OMAR to undertake any writing tasks, and 
there is an admonition that final responses cannot be formulated 
until all the testimony is taken into account. 

3. Stakeholder presentations . Although there may be some 
spill-over of expert presentations from tl:s first day, the morning of 
the second day is intended primarily for shorter presentations from 
persons and groups having a particular point of view toward the 
topic. These may be spokespersons for the various industries or 
vendors who have some involvement in this area. There will usually 
also be representations from disease organizations, e.g. American 
Cancer Society, pacient advocacy groups, the legal profession, and 
medical ethicists. In theory, anyone is entitled to make a 
presentation in this forum. It is also at this point that some of the 
most heated controversy is joined. 

4. Question periods and open d* .cussions . Question periods 
are dominated by the panelists who sit as a group on stage, 
appearing and acting somewhat like the Supreme Court or a 
Congressional committee. Some questions are also entertained from 
the floor. Most of these come from other speakers and there are a 
scattering of questions from other members of the audience. 
However, most of the audience most of the time are merely listeners 
and observers. At least until the time of this writing no attempt 
has been made to introduce remote hook-ups so that observer- 
listeners can participate from other parts of the country although 
the technology for such remote linkage, both audio and video, has 
been available for some years. 

5. The drafting session . The most dramatic phase of the 
conference takes place behind closed doors starting the afternoon of 
the second day and extending sometimes non-stop until dawn of the 
third day. This is the writing session. The goal of this session is 
clear, compelling, and final: the panel must prepare a text 
responsive to all the pre-set questions and representing a consensus, 
i.e. an essentially unanimous rendering of their views, to be 
distributed to all conferees at 9:00 a.m. of the third and tmal day. 
It is up to each panel to manage its own time and divide up the 
writing tasks as it sees fit within certain constraints established by 
OMAR. Thus the role of the chairperson at this stage is crucial. 
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All statements are prepared in four drafts, color-coded to mark 
the progression toward the finished state. The OMAR support 
contractor proces es hand-written material from panel working 
groups very rapidly so that there is minimal waiting for typed text. 
Usually, an entire draft will be discussed by the total group, 
question-by-question. Then recommended changes and additions will 
be made by assigned individuals or subgroups, followed by a 
reconvening of the ppnel as a whole. 

With the exception of a 2-3 page draft introduction to the 
topic, none of the actual writing is done in advance by OMAR or 
the sponsoring BID. However, a handful of OMAR and BID staff 
members sit in as observers and occasionally make suggestions or 
comments on the wording of responses, usually for clarification. As 
the custodian of the process the OMAR staff is concerned that the 
panel adhere to the ground rules which include basing all statements 
only on evidence presented either orally or in written form and 
concentrating on technical as distinct from moral, legal, or fiscal 
issues. Cost-benefit analysis in either a broad or narrow sense is 
generally out-of-bounds for a consensus statement. 

The writing sessions are almost always long and exhausting but 
also perceived as uniquely rewarding by most panelists. Usually they 
go longer than panel members anticipate, despite advance notice 
from OMAR that most writing bessions are very long and arduous. 
It is unusual for a panel to complete its work before midnight. The 
norm is somewhere between one and three a.m. 

Following completion of the panel session, the OMAR support 
contractor must still type and reproduce the semi-final draxt for 
distribution to as many as 400 persons by 9:00 a.m. 

6. Public review of the draft . The panel chairperson begins 
the last plenary session by reading the draft text. At the close of 
the reading, questions, comments, and suggestions for amended 
wording and phrasing are entertained from the floor. At this time 
there may be little response from the panel. In effect the 
suggestions are taken under advisement without clear indication c f 
how they are to be handled. 

7. Final closed review . At approximately 11:00 a.m. of the 
third day the panel retires for a final private session to amend the 
document, taking into account whatever comments from the floor 
they feel are important. There is great time pressure in this 
meeting which can last no longer than one hour. At the end of the 
meeting a final draft including final alterations of text in 
handwriting is photocopied for distribution at the press conference. 

8. Press conference . A press conference is convened at 
noon to make public announcement of the findings of the panel. 
The panel chair is in charge of this meeting. There is usually an 
initial summary statement of one or two paragraphs which is read by 




the chair. Then questions are entertained on any aspect of the 
deliberations. Sometimes the chair responds to all questions but 
more commonly different members of the panel respond, depending 
on the role they played in the drafting or the knowledge they may 
have regarding the subject. These responses may go beyond what is 
said in the actual document. 



Attendance at the press conference varies depending on the 
perceived news value of the topic. Thus, narrow and technical 
topics are likely to draw only a few specialty reporters, whereas 
relatively hot topics like AIDS screening, or topics with broad public 
interest, like obesity will, draw dozens of reporters and several 
network and cable camera crews. 



Post-conference Activity . The CDC process does not end with the 
conference. There are several follow-up activities for each conference 
intended either to enhance its impact or to maintain the integrity of the 
process. These can be described under four categories: (1) diffusion to 
medical practice, (2) diffusion to the public, (3) evaluation, and (4) 
follow-up on the topic. 

1. Diffusion to medical practice . Following an additional 
round of editing by the panel chair, the statement is printed in two 
versions, a "flat," a plainly printed documentation standard size 
paper, and somewhat later a booklet with a glossy colored cover. 
The flats are printed by the Government printing Office and a run 
is typically 25,000. The printing run for the pamphlet version is 
typically 5,000. For some topics, private sector underwriters support 
additional printing runs and popular CDC statements are sometimes 
reprinted. 

An effort is also made to publish the statement in at least one 
journal which is widely read by medical practitioners. Right of first 
refusal is given to the Journal of the American Medical Association t 
the most widely read medical journal. In recent years JAMA has 
a-most always accepted the statements for publication. 

Since early in 1984 OMAR has also made arrangements for the 
Hospital Satellite Network (HSN) to videotape segments of the 
conference and interviews with panelists as a package for Continuing 
Medical Education credit. The one-hour tape is then inserted in the 
HSN programming schedule and broadcast to its 500 subscribing 
member hospitals. The broadcast is repeated at least three times in 
different time slots to allow maximum opportunities of exposure. 

2. Diffusion to the public . Although medical practitioners 
are the prime target for CDC's, there is also a clear desire to reach 
a broader audience. This is the prime rationale for the press 
conference. There are also press releases before the conference 
which go to 500 editors around the country. OMAR has a full time 
communications officer who handles public relations and distribution 
generally and maintains contact with both print and television media 
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representatives. Each Institute of NIH also maintains an office of 
this kind and assigns its public relations officer to work closely 
with OMAR on conference publicity. 



Although there are no formal data available on actual news 
coverage, conferences with topics of broad interest seem to be fairly 
well covered by the major national newspapers and the wire 
services. Hot topics may also get some brief mention on nightly 
national network news. For example, the CDC on osteoporosis 
which concluded on February 11, 1S87, received a two minute special 
segment on the NBC Nightly News the same evening. More 
typically, CDC*s are given fairly expansive coverage on local 
Washington, D.C. news programs. 

3. Evaluation . There is no institutionalized procedure for 
evaluating conferences. However, various types of data are collected 
routinely, and this data is sometime - summarized and reviewed by 
OMAR. A minimal source on all attendees is the "door survey 11 
distributed at the beginning of the conference which asks for data 
on occupation, place of work, educational level, reason for 
attending, and source from which information about the conference 
was obtained. A second source is the post-conference reaction form 
required of all attendees who desire CME credit from the NIH 
Clinical Center. This form elicits general evaluations of the 
conference, the speakers, and the statement. It is in no way a test 
of the attendee's understanding of the topic or learning from the 
process. Data is tabulated and internal memoranda are occasionally 
prepared based on these two sources but such findings have never 
been published. 

The door survey results indicate that the conference is largely 
attended by medical practitioners and other related professionals in 
appropriate numbers, with a fairly heavy proportional participation 
from the local area and from persons employed by the federal 
government in one capacity or another. The CME surveys generally 
show very positive ratings of all aspects of the conference on which 
they are questioned. 

A more detailed source of feedback to OMAR is a 12 item 
post-conference questionnaire which all panelists are requested to 
return. This form asks for both ratings and comments on all 
aspects of the conference, starting with the background material, 
abstracts booklet, the questions posed, and moving on to each 
element on the agenda. OMAR has also made periodic tabulations 
and summaries of this information. Some publication of this material 
was planned for 1987. The panel s;urvey results are treated as 
strictly confidential and are closely guarded by OMAR staff. 
However, the trend of responses over the years has been clearly 
very positive on most aspects. Much of the comment and concern is 
focused on the writing process and the long writing session which 
many panelists find gruelling. Although many suggestions have been 
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made of ways to improve this process, there is no consistent 
feedback pointing to clear alternatives or significant modifications. 

Approximately a month after each conference, OMAR convenes 
a post-mortem meeting involving members of the planning group (but 
not the chair)* At these meetings observations are made about all 
aspects of the CDC including the observed performance of the panel 
and the panel chair. The purpose of the review is primarily to 
make sure that the CDC process remains on track. 

In addition to such routine data collections and informal 
evaluations, NIH has commissioned three evaluation studies over the 
years. The first was conducted by the University of Michigan and 
focussed primarily on the conference process (Wortman et ah, 1982), 
The National Opinion Research Center, together with CDP 
Associates, the prime OMAR support contractor, conducted a survey 
of dissemination effectiveness resulting from two conferences ; .i 1981 
and 1982 (on CT scanning and on total hip joint replacement), 
Results of these surveys (reported by Jacoby, 1983) indicated that 
awareness levels after a conference ranged from 12% to 37% among 
nationally targeted medical specialties. However, less than half of 
those who were aware of a conference had any knowledge or recall 
of conference findings. 

In the early 1980's NIH commissioned a much larger study of 
CDC impacts from the Rand Corporation. Results of this study have 
not yet been published. 

4. Follow-up on the topic , NIH has no established process 
for pursuing a topic beyond the conference, Therefoi^, as new data 
become available which would contradict or bring into question panel 
conclusions, there is no way to amend the statement. No panel has 
ever been reconvened although some topics have been revisited over 
the years (e.g. breast cancer, three times, and pain, twice, each time 
with a slightly different slant). 

No specific efforts are mads by NIH to follow up on any 
recommendations or findings contained in the statement or 
articulated by the panel. In the late 1970's there was some 
consideration given to following consensus conferences with what 
were tentatively referred to as "interface development" conferences. 
These later events would pick up where the CDC's leave off on some 
important topics, directly treating the social, ethical, legal, and 
economic implications. Such conferences were intended to be within 
the province of a non-NIH unit of the Department of Health and 
Hu.aan Services, the National Center for Health Care Technology, 
The NCHCT had a brief life in the DHHS, being abolished in the 
early 1980's as part of a budget-cutting exercise. 
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It should be further noted that NIH does not follow up in any- 
way on the research implications which are always the last item in a 
statement, this despite the fact that NIH is primarily a research 
supporting government agency. 

Costs 

The average cost of a CD conference has been reported in one 
source (Henig, 1985) as $75,000. However, this figure only covers direct 
charges for the work of the OMAR support contractor including 

I participant reimbursements and printing. There are much larger hidden 

costs including an OMAR professional staff of six, devoted nearly full 
time to conference-related activity and substantial labor contributed by 

j the staff of the particular NIH bureau or Institute sponsoring a 

conference. OMAR also calls upon the services of the National Library, a 

j graphics unit, the GPO, press offices, and others, and does not pay for 

the use of the conference hall. Videotaping for the HSN has so far been 
underwritten by private sources as have some printing runs for some 
conferences. Adding to these items the necessary administrative and 
overhead costs of the support contract, the real, costs of a conference 
are probably nearer to $200,000 although nc such accounting has ever 
been made. 

Above and beyond those costs are the very substantial contributions 
of the panel and speakers, all of whom donate their services. Because 
most of these persons are MD's or Ph.D.'s at the top of their professions, 
their total contribution in 1987 dollars might be as much as $100,000, 
especially considering the amounts of time invested by speakers in 
I preparing presentations. At first glance, such contributions might appear 

to relieve the government of a substantial cost burden. However, when 
such persons donate their time, they are not forfeiting a day's pay. In 
fact, their institutions or their patients are paying through fees and 
salaries. The source of such funds in many cases can be traced back to 
the federal government in one way or another. 

Weaknesses of the Model 

The Consensus Development Conference model has been subjected to 
occasional sharp criticism from participants representing minority or 
divergent views (see, for example, Ahrens, 1985, Oliver, 1985a, reply by 
Jacoby and Rose, 1985 with rejoinder by Oliver, 1985). There is no 
question that there is considerable pressure during the writing session to 
submerge differences and disagreements. Statements are not intended to 
j be literature reviews or scholarly Analyses which reflect divergent 

opinions. 

There is also strong pressure exerted against any panel member 
issuing a minority report or even a minority view on a given question. 
In this sense it is 1 ike a jury proceeding. There are at least two 
consequences of this which are unfortunate. First, the language of a 
statement which is acceptable to all panelists may sometimes blur rather 
than sharpen important issues. Secondly, there is a tendency not to 

67 

ERLC 74 



select issues which are too controversial in the first place, regardless of 
their apparent importance* In effect, NIH does not want any CDC 
reporting that there is not any consensus on an issue even though this 
could be an important fact worthy of wide dissemination. 

Ahrens and Oliver also charged that the pp.nel selection process was 
stacked in favor of one point of view and that their conclusions were 
fore-ordained. This is not a fair criticism of most panels at least from 
the observations made by this author. As stated earlier, panel selection 
is a delicate process in which expertise and objectivity are sought by the 
planning committee. The OMAR staff is particularly sensitive to this 
issue and are likely to challenge some panel selections if it turns out 
that the proposed panelist has a previously stated position on the 
conference topic. 

The CDC process does not work very well when the data base is 
weak, i.e. when few empiri al studies have been addressed to the problem. 
CDC's are not designed to explore new ground nor to give a thorough 
airing to a previously obscure issue. 

It is also a less-than-adequate mechanism for handling topics which 
are very complex, and/or where the data is substantial but diverse and 
inconsistent. A 1986 conference on the management of pain, for example, 
had difficulty coming up with a statement which had much force because 
the defined topic was too broad and many-faceted. 

Strengths 

In spite of such criticisms, the CDC process has much to recommend 
it in the medical context. It is an especially effective tool for focusing 
attention on a topic. Regardless of what the statement says, the fact 
remains that many hundreds of people including leading experts and 
medical opinion makers have concentrated their attention on this one 
issue for a three-day period, some of ihem much longer. They have 
clarified their views, shared them with one another in an open forum, and 
either changed or reinforced their thinking on the matter. 

Secondly, the CDC process usually succeeds in creating a document 
with both high visibility and high credibility. The care with which 
panelists and speakers are selected as well as the emphasis on the special 
structure of the event contribute to this credibility. 

The pressure to create a concise and readable statement with 
minimal use of technical language also seems to be a desirable feature, 
even though some fine points and disagreements are lost or muted. 

Finally, the attempt to force consensus among a diverse set of 
experts and stakeholders succeeds much more often than it fails, and it is 
therefore very much worth the effort. 
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Variants 



The conference process described above has beer maintained with 
remarkable consistency for nearly a decade since its inception. With such 
a record, and especially with growing acceptance of the process within 
the many Institutes, Bureaus, and Divisions which make up the NIH, there 
is an understandable reluctance to tinker with the process in spite of 
various criticisms and manifest weaknesses. Up to the present, thsre has 
been only one significant variant to the CDC model, and this is the 
process put forward by the National Institute for Child Health and Human 
Development. The NICHD model puts much more emphasis on extended 
panel deliberations and the preparation of a comprehensive and scholarly 
monograph to accompany the statement, and to stand as a publication in 
its own right. NICHD panels have planning meetings, take expert 
testimony, and do some of their writing long before the conference, is 
convened. This may reduce the burden on the final writing session but 
clearly increases the over-all panelist burden and greatly increases 
conference costs. Although NICHD spokespersons are strong advocates of 
their variant (Hill, 1986) there is no evidence to indicate that it has 
greater impact or is more cost-effective than the standard model. 

As stated earlier, conferences explicitly imitative of the NIH model 
have been undertaken in the United Kingdom and Sweden. The 
substantial difference in these European approaches has been the inclusion 
and even the highlighting of social and ethical aspects of a topic which 
are ruled out-of-bounds by NIH. This type of emphasis suggests that the 
European variants are more concerned with influencing health practices 
and public attitudes than in influencing medical opinion as such. 

Implications for Education and Other Fields 

In many respects the NIH model reflects the unique structure of 
medicine and the dominance of one professional group, the MD's, in all 
medical research, technology development, policy and practice. Over the 
years the NIH has gradually gained acceptance in the medical community 
as a respected source of technical knowledge as well as the prime 
supporter of all kinds of medical research, but there is great reluctance 
by the NIH and the community it serves to extend its role beyond 
research. The NIH is not supposed either to make policy or to regulate 
practice. This is the major reason why social, ethical, legal, and fiscal 
issues are skirted. NIH, which is itself dominated by MD's who hold most 
of the same norms as their colleagues, is very cautious in extending its 
role into new areas such as the arbitration of conflicts, even of a 
technical nature. 

Yet for most other fields including education, no costly conference 
proc Mure could be justified if all such issues were held in abeyance a *d 
it is doubtful that participants would have the self-discipline to restrain 
themselves in this regard anyway. 
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The credibility of the medical CDC's also depends to some extent on 
what might be called the invisible collegial structure. There is a very 
strong norm in medicine on the publication of all empirical findings in 
strictly refereed journals. Furthermore, there is a distinct hierarchy 
among journals with the New England Journal of Medicine being clearly 
the most prestigious. Thus when even a small handful of experts sit 
down to plan a meeting, they already know the people who are doing the 
r ost important work in a given field. Furthermore, they are likely to 
have high levels of agreement or consensus on who these people are. As 
a consequence, even though some speakers (e.g. Ahrens and Oliver) may 
feel that their views are not adequately reflected in a CDC statement, 
they are nevertheless included in the proceedings and allowed to make 
theij. case. The question, therefore, arises as to whether a field like 
education is organized similarly with norms as distinctly shared and 
perceptions of expertise as clearly focused and consensual. 

Another important factor is the credibility of professional expertise 
among the public at large. Although the medical profession no longer has 
the near immunity from public criticism that it once enjoyed, the public 
and its elected representatives still is rather deferential to experts with 
medical credentials, especially when it is dealing with apparently technical 
issues. This is not true of experts in many other fields, particularly 
education. On many educational issues, members of the public often 
accept no expertise as greater than their own "common sense" opinions 
and perceptions. It is thus doubtful that a conference organized along 
the same lines as a CDC would enjoy the same credibility. 

Finally, we come to the question of costs. All conferences are 
expensive but the elaborate planning, structuring, and dissemination 
activities surrounding CDC's put them in a class by themselves. As noted 
earlier, the real costs of a typical CDC probably range between $200,000 
and $300,000, especially when "donated" time is included. Medical 
practitioners and researchers may be able to donate so much time because 
they are very well paid to begin with and have substantial control over 
their own calendars. Prevailing conditions in othei fields are likely to be 
very different, yet the resources available to support conferencing 
activity are likely to be more modest. 

We might then want to ask if there are cheaper ways to run 
consensus conferences to produce results nearly as satisfactory. The 
number of potential variants in conference structure is infinite so the 
answer is probably *yes' in theory, but we have no idea what features are 
most dispensable. One very important aspect of the CDC's is the 
continuity in the process, the ability to sustain a well-articulated 
conference model over a great number of topics and over a ten-year 
period. It takes reliable resources to do this. It also takes an 
experienced team of conference monitors and managers such as OMAR has 
developed with its support contractors. Some of the early years were 
rough for OMAR, and there was a period of nearly two years when hardly 
any CDC's were undertaken. Anyone contemplating the start-up of a 
CDC process in another field should also expect to have these growing 
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pains. They should further realize that the model that will eventually- 
work for them may not end up looking much like its NIH antecedent. 

What, then, are the features of the CDC process which are most 
valuable as exemplars for other fields? In the opinion of this author, the 
focus on "consensus" as such is not one of these features. We have seen 
that "consensus" is often problematic even on technical medical issues 
unless they are narrowly defined. In education we might also be able to 
define a topic so narrowly that a kind of consensus would be achievable, 
but then the interest in the topic would probably be too limited to justify 
the investment in the conference. 

On the other hand, what is especially useful as an example for other 
fields is the emphasis on developing and maintaining a consistent process, 
including planning carefully for stakeholder and expert participation, 
providing advance materials of high quality, structuring sessions so that 

the meeting as a whole has a focus, and forcing a statement from a panel 
which represents a collective view and a product that can be circulated. 

The CDC should be viewed as an interesting and probably important 
type of social engineering. We know what it is, and we can replicate it 
with predictable results at least within one field, medicine. This is no 
small achievement. Some day, constructive conferencing may become a 
kind of science. There will not be just one model but many clear 
alternative conference models which produce different kinds of results in 
a similarly predictable fashion. When that day comes, we will look back 
on the NIH model of the 1980's as an important early step in the 
emergence of this very useful branch of applied social science. 




71 



78 



REFERENCES 



Ahrens, E.H. Jr. "The Diet-Heart Question ia 1985: Has it Really Been 
Settled?" The Lancet . May 11, 1985: 2085-2087. 

Henig, R. M. "The Process and Provocations of Consensus." Medical 
World News . October 14, 1985: 96-111. 

Hill, J. G. "Technology Assessment at the National Institute of Child 
Health and Human Development: An Alternative Model." in H.M. 
Wisniewski & D.A. Snider, Mental Retardation and Developmental 
Disabilities: Research, Education, and Technology Transfer . New 
York: Annals of the New York Academy of Sciences, 1986. 

Jacoby, I. "Biomedical Technology: Information Dissemination ^nd the 
NIH Consensus Development Process, " Knowledge: Creation, 
Diffusion, Utilization , 5, December 1983: 245-261. 

Jacoby, I. and M. Rose. "Consensus Conference on d.oiesterol and Heart 
Disease (letter reply to Ahrens, 1985, and Oliver, 1985a) The Lancet . 
July 27, 1985. 

Guidelines for the Selection and Management of Consensus Development 
Conferences . Bethesda: Office of Medical Applications of Research, 
National Institutes of Health. May, 1983. 

Participants 1 Guide to Consensus Development Conferences . Bethesda: 
Office of Medical Applications of Research, National Institutes of 
Health. 

Oliver, M. F. "Consensus or Nonsensus Conferences on Coronary Heart 
Disease, The Lancet . May 11, 1985(a): 1087-1089. 

Oliver, M. F. "Consensus Conferences (letter, rejoinder to Jacoby & 
Rose) The Lancet . August 17, 1985(b): 389. 

Perry, S., and J.T. Kalberer, Jr. "The NIH Consensus-Development 
Program and the Assessment of Health Care Technologies: The First 
Two Years." New England Journal of Medicine . 303: July 17, 1980: 
169-172. 

Wartman, P. M. et. al. Evaluation of NIH Consensus Development Process 
~ Phase 1: Final Report . Ann Arbor, MI: Institute for Social 
Research, University of Michigan, Sept. 28, 1982. 



ERLC 



72 

79 



DISSEMINATION, COMMUNICATION EFFECTS AND 
ORGANIZATIONAL EFFECTIVENESS 



Kathryn Theus 
Associate Director for Communication and Administration 



National Center for Postsecondary Governance and Finance 




73 



SO 



DISSEMINATION, COMMUNICATION EFFECTS AND 
ORGANIZATIONAL EFFECTIVENESS 



It isn't enough to discover a truth or to create a fact. Nothing of 
substance changes unless others learn the discovery is worth their 
attention. It is incumbent upon researchers, then, to attack the problem 
of access to research findings with the same seriousness with which they 
approach the formulation of theories and research methodologies. 
Otherwise, their work, like the hypothetical tree that falls unheard in the 
forest, will not only go unnoticed, but philosophically speaking, may not 
really exist. 

Mason (1986) and others throughout this monograph, have reminded 
us that dissemination is a major function of all research institutions. 
The Proceedings of the Conference on Dissemination Findings of Research 
on Postsecondary Governance and Finance , often conveys an impression of 
dissemination as an arrow let loose toward a target. The arrow point 
carries understandable and timely accounts of Center research findings. 
The target represents selected audiences. The shaft corresponds with 
appropriately chosen interpersonal and mass media channels. 

The analogy fits the classical conceptualization of communication as 
set forth by Shannon and Weaver (1949). They identified five parts to 
the structure of information transfer: source, transmitter, channel, 
receiver and destination. This essentially one-way model inspired decades 
of research on the effects each component has on cognition and attitude 
or behavior change (Hovland, 1959; McGuire, 1969; Schramm et. al., 1971; 
McGuire, 1973). 

The purpose of this paper will be to remind researchers and those 
they engage in the archery of information transmission that dissemination 
does not constitute communication. It will draw attention to the 
importance cf planning dissemination activities around knoun effects of 
communication with target audiences. It also will show that evaluations 
are necessary in establishing that communication has made a difference 
among audiences and in establishing that dissemination activities 
contribute to organizational effectiveness. 

Communication Effects Research 

Why is it that research center dissemination specialists frequently 
assume that their products succeed in communicating information and in 
changing attitudes or behaviors among members of target audiences? 
Researchers know that dissemination is not the same thing as 
communication — that there appears to be no direct causal link between 
providing information to target audiences and affecting changes in 
audience comprehension, attitude or behavioral adoption. That is to say, 
inconsistency in cause and effect provides the backdrop for much that 
can be said about many specific communication outcomes. 
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Nevertheless, scholars have viewed effects of communication 
activities probabilistically in predicting whether they will be long-term or 
short-term; deliberate or unintended; directly causal or interactive; affect 
individuals, groups, organizations or larger social groupings; and/or will 
be primarily reinforcing, contribute to minor change or to major changes 
such as conversion. (Klapper, 1960; McQuail, 1983). 

Researchers generally expect that short term, deliberate effects may 
be anticipated from well-developed campaigns using several media to 
obtain cognition development and to persuade selected populations. 
Examples incl*' 'e product advertising, fund-raising, public-information 
programs and political campaigns. These campaigns are overt in their 
objectives, are operated in a limited time span, have legitimate and 
authoritative sponsorship and are consistent with consensual values of 
established institutions. Non-deliberate short-term responses also might 
be precipitated in the form of collective reaction, which could include 
fear, anxiety, or social upheaval* (Ehling> 1987) 

Those seeking long-term deliberate effects, such as diffusion in 
development (diffusion of innovation) and knowledge distribution (Rogers 
et al. 1973) recognize that these are achieved through interpersonal 
networks, power structures and formal education. Researchers know that 
mass communication plays only a supportive role in achieving long-term 
effects. Unintended long-term communication effects may include 
socialization, social control, reality definition and institutional change. 
The longer-term the effect, the greater will be the cost in achieving it. 

The problem for dissemination specialists is one of balancing costs 
with desired effects and determining what effects can be obtained within 
known constraints. McGuire (1972) suggested a framework for studying 
effects that combines Shannon's model as independent variables with oix 
of his own categories as dependent variables. Those categories include: 
presentation - being presented with communication; attention - attending 
to it; comprehension - comprehending its content; yieldin g - yielding to 
it; retention - retaining this new position; and acting acting on the 
basis of it. In planning communication, a dissemination specialist could 
ask how each variable on the horizontal axis contributes to each variable 
on the vertical axis (see Fijj. 1). If nothing else, such an examination 
would tend to temper their estimates of anticipated effects. 
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Figure 1 

MCGUIRE'S CXDMMUNICATION-PERSUASION MATRIX 

Source Message Channel Receiver Destination 



Presentation 



Attention 



Comprehension 



Yielding 



Retention 



Action 



Rogers (1971, 1983) developed a set of similar categories in studying 
the diffusion of innovation, identifying five stages in the adoption 
process: 1) knowledge 2) persuasion 3) decision 4) implementation and 5) 
confirmation. Hundreds of studies have shown the adoption process to be 
tied to interpersonal communication, yet affected by the social attributes 
of the potential adopter, the social environment, the psychological 
characteristics of the adopter, and the attributes of the innovation, itself, 
which may speed or retard adoption. 

Earlier, Katz and Lazarsfeld (1955) studied the role of mass 
communication's support of interpersonal networks, likening the 
interaction to a "two-step flow" of information. Mass media act as 
influencers of opinion-leaders who mediate message flow to less active 
members of audiences. By the early .970s McCombs and Shaw (1972) 
reconceptualized this idea by calling mass media "agenda setters," e.g., 
mass media tell people what to think about, not what to think. 
According to this model, communication with target audiences may be 
enhanced as opinion leaders consider and respond to agendas set by 
media. 

Along these lines, Katz (1974) and associates began to think of the 
message recipient as the active agency in deciding what medium to use 
and what messages to attend to, dependent upon the specific gratification 
or use sought. In other words, the utilization of mass communication 
became the effect and the gratification became the cause. Psychographic 
studies have arisen out of the presumption that if audience needs can be 
identified, then messages can be tailored in ways that will encourage 
specific audience response as information-seeking and other behavioral 
activity. 
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Public relations theorist James Grunig (1979, 1983, 1984, 1987), 
operating out of a similar context, noticed that publics with similar 
demographic characteristics often do not manifest similar responses to 
communications. His research showed that attention to communication is 
often situationally defined. Dependent upon levels of problem recognition, 
constraint recognition and involvement in issues, individuals (and publics) 
will passively process or actively seek information. 

Grunig defined publics (in keeping with Dewey, 1927) by the degree 
to which aggregations of individuals are affected by (latent publics) and 
recognize a problem (aware publics), and organize to do something about 
it (active publics). Hence, as situations change, the composition of 
publics also changes. Aware and active publics are more likely to seek 
and process information from media, while non-publics "(those not affected 
by an issue) will process information primarily from passive media such as 
television or radio. Passive information processing eventually may lead to 
awareness and information-seeking. 

Communication campaign researchers (McNamara, Kurth and W^r^en, 
1981) have found that three supplemental activities often help 
communication activities make an impact on public cognitions, attitudes 
and behaviors. The three include education, enforcement and engineering, 
described as the three-E model. education in a public communication 
campaign serves to inform people about the important details or facets of 
the issue. Enforcement serves to enforce lega 1 or other constraints on 
undesirable behaviors, and engineering serves to structure the physical 
environment to make it more difficult to behave in an undesirable way. 
These mechanisms might be important in translating policy research 
findings into policy applications. The three-E model has been put to use 
in campaigns to prevent drunk driving, to require seat belts, and to 
discourage smoking. 

Stud ; 3s (McAlister, 1981) have also shown that interpersonal support 
provides an essential backup to changing behaviors where engineering and 
enforcement are not possible, particularly in health- or safety-related 
activities. Organizations such as Alcoholics Anonymous or Weight 
Watchers are examples of such support groups. 

Communication Effects Research as a Management Tool 

If the findings from most communication erfects research are 
correct, the most likely effect of communication programs will be 
cognitive (awareness or understanding). Attitude and behavioral change 
(in that order) are more difficult to affect. These may be modified only 
after long-term, multi-faceted approaches to communication have occurred 
on salient issues that individuals have recognized as problematic and 
about which they have resolved to do something. 

The dissemination strategist for a research center, to be an 
effective communication manager, must help researchers to define the 
nature of the desired outcomes of their communication about research 
findings (e.g. long- or short-term, intensity, direct causal, degree). The 
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manager must help researchers to identify audiences as real (e.g. latent, 
aware or active) as opposed to imaginary (non-publics) and to select 
communication approaches most suiter 4 for reaching them (e.g. 
interpersonal versus mass mediated, .agineered, enforced or some 
combination) as defined by their levels of information-seeking or 
processing behaviors and by the structures available for providing 
constrained or coerced effects. 

With communication objectives specified, the evaluation of the 
effectiveness of communication becomes less problematic. If the goal of 
communication in a research center is to transfer information to groups 
of opinion leaders, then it will be possible to show how dissemination has 
contributed to that information transfer by measuring information change 
among them. If the goal is to affect policy change and other behavioral 
adoptions, this can also be inferred with careful aggregation of evidence. 

But without the planning of communication by setting objectives and 
then analyzing their potential effects and costs (e.g., conducting 
formative and evaluative research), communication becomes a haphazard 
enterprise and its effects, if any, will be accidental. Dissemination 
efforts are limited by the degree to which disseminators fail to see the 
organization's communication as a measurable exchange between the 
organization and the publics in its environment. 

Linking Communication to O rganizational Effectiveness 

A word about communication effects research as a measure of 
organizational effectiveness: Throughout this monograph authors have 
asserted that the Center's dissemination activities should contribute to 
attaining its organizational goals as stated in terms of cognitive, 
attitudinal and behavioral impacts on policy-makers and researchers, and 
through these, impacts on policy and research. Varieties of dissemination 
activities have been su& jested to engage target audiences in the 
integration of research findings as cognitions and as behavioral 
applies t"' ns. While these activities reflect logical approaches, they do 
not easLy translate into measures of organizational effectiveness. 

Of course, definitions of organizational effectiveness are fraught 
with inconsistencies. Some studies define effectiveness as goal attainment 
(Georjjopoulous and Tannenbaum, 1957; Molnar and Rogers, 1976; Goodman 
and Pennings, 1977). Others employ a systems approach, which suggests 
that effectiveness occurs as organizations attain environmental 
equilibrium, rather than entropy (Katz and Kahn, 1966; Yuctman and 
Seashore, 1967; Molnar and Rogers, 1976). Yet others identify 
effectiveness as satisfaction of major constituencies (Bennis, 1966; Katz 
and Kahn, 1975). At a macro-level, each of these definitions speaks to 
organizations achieving some degree of environmental autonomy as 
organizations seek to protect their independence, which Kage (1980) 
suggests is derived as organizations employ strategies of domination or 
interdependence. 
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Whether these or other definitions obtain, the key to ascertaining 
effectiveness depends upon some formal means of assessment. Such 
assessments might hinge on at least one quality each perspective has in 
common* For example, measures may focus on the means by which 
effective organizations seek and exercise a degree of autonomy in 
reaching goals, in managing inputs, processes and outputs, and in dealing 
vith internal and external cor, tituencies (Cameron and Bilimoria, 1985), 
Measures may be operationalized as less regulation by government or less 
pressure from internal cv external interest groups or the decrease in 
costs of managing those influences. 

Organizational theory suggests that communication programs 
contribute to organizational autonomy. Grunig (1987) points out that just 
as organizations may employ strategies of dominance or cooperation in 
seeking to attain autonomy in the environment, organizations may employ 
communication strategies in parallel ways. Asymmetrical organizational 
communication seeks to dominate publics and other groups in order to 
achieve attitude change (agreement) or behavioral modification. This 
occurs when organizations research attitudes and opinions of target 
groups and strategically tailor messages to appeal to their mindsets. 
Organizations that seek to cooperate with publics by managing 
interdependence set symmetrical objectives for communication such as 
message retention (accuracy) and acceptance of cognitions 
(understanding). The latter approach characterizes communication 
objectives that emphasize mutual understanding, an approach often used 
by non-profit organizations and other organizations that tend to have 
ambiguous goals, amorphous linkages with the enviror ~ r . and a variety 
of constituencies. 

Thus, if a connection between effective cemmr nidation and 
organizational effectiveness may be inferred through the dimension, 
autonomy, then in developing communication programs dissemination 
specialists will seek to achieve effects that will contribute to autonomy. 
For example, research shows that communication campaigns may reduce 
opposition to issues but may not produce community or government 
support for them. Communication programs also may not increase 
employee morale or productivity, although they may help employees better 
coordinate their worK with others in the organization (Grunig, 1987), 
And the author's research seems to indicate that open communication 
with media about specific and measurable organizational goals will result 
in greater accuracy of reporting (and in ways more salutary to 
organizations). Non-profit organizations often complain of misunder- 
standings with media because, whether or not it is deliberate, they do not 
typically- embrace such openness nor set such standards of performance. 

Each of these outcomes of communication TTtay contribute to an 
organization's autonomy in its environment. The key to establi&mng the 
contribution of communication to organisational effectiveness resides in 
the quality of the communication effects re- jarch that communication 
managers design. Both formative and evaluative research are essential. 
Formative research assists managers in identifying problems that affect 
publics and helps in planning communication programs to reach them. 
With formative research, dissemination specialists can advise 
organizational management on the probable impacts that communication 
efforts may have on publics (or the consequences that publics may have 
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on the organization). Methods may include opinion research, 
environmental scanning, issue analysis, stakeholder analysis, or 
communication audits. Exploring linkages the organization has with the 
environment often helps to identify its latent, aware or active publics. 

Evaluative research will base its measures on the steted objectives 
of dissemination activities as part of an overall communication goal. 
There is little to be gained by counting the number of press releases sent 
out, audience members contacted, meetings held or publications issued, if 
these are to be considered principal measures of successful 
communication. While these do provide insights into information available 
in the environment about the • organization and are valuable for that 
purpose, they do not measure the impact of dissemination in any 
meaningful way. Grunig calls these "process evaluations." 

Impact can only be measured with effects evaluations on variables 
such as awareness, knowledge, understanding, attitudes or behavior, 
through survey or experimental research. Effects evaluations can be 
quite costly, however, and many organizations do not utilize effects 
evaluations because of their cost. But process evaluations combined with 
periodic effects evaluations can infer impact. For example, an 
experimental procedure may demonstrate an effect and the percentage of 
people affected in the experiment may be inferred as a percentage of 
people contacted by the dissemination activity* Such combined approaches 
*,o evaluation can be used to approach the issue of a communication 
program's contribution to organizational effectiveness, by logical 
deduction rather than by direct measurement. 

Conclusion 

A research center's effectiveness obviously will be related to the 
significance of its research findings. A research center such as the 
National Center on Postsecondary Governance and Finance is better 
positioned than other kinds of organizations to recognize the importance 
of evaluative measures as evidence of tho strategic management of the 
organization's primary mission — creating and communicating research 
findings that make a difference to researchers and policy-nakers. 

Dissemination specialists know that it isn't enough to discover a 
truth or fact. It is the their role to make the truth available to those 
for whom th~ truth matters. As for other publicly or privately funded 
research centers, communication impact reports will assist the National 
Center in making the case to its sponsorship that research findings have 
found their way into the hands of people who can use them. Such 
reports will furnish concrete evidence of how a successful communication 
program enhances the Center's autonomy in its environment, and this 
evidence may be used as a measure of the Center'*,? effectiveness as a 
research organization. 



ERLC 



80 

87 



REFERENCES 

Bennis, W. Changing Organizations: Essays in the Development and 
Evaluation of Human Organizations , New York: McGraw Hill, 1966. 

Cameron, K. "Critical questions in assessing organizational effectiveness." 
Organization Dynamics , 9(1980), pp. 66-80. 

Cameron, K. and Bilimoria, D. "Assessing Effectiveness in Higher 
Education." The Review of Higher Education . 9(1985), pp. 101-118. 

Dewey, John, The Public and Its Problems . Chicago: Swallow Press, 1927. 

Ehling, William P. "Effects of Mass Communication and Public Relations 
Practice." in J. Grunig (ed.) Excellence in P ublic Relations and 
Communication Management: Contribution s to Organizational 
Effectiveness . New York: Lawrence Earlbaum, in press. 

Georgopoulos B. and Tannenbaum, A. "A study of organizational 
effecth eness." American Sociological Review . 22(1957): pp. 534-540. 

Gerogopoulos, B. and Mann, F. The Community General Hospital New 
York: McMillan and Co., 1962. 

GLorpade, J. (ed.) Assessment of Organizational E ffectiveness: Issues, 
Analysis, and Readings . Pacific Palisades, Ca: Goodyear, 1971. 

Goodman, P. and Penniags, J. New Perspectives on Organizational 
Effectiveness . San Francisco: Jossey-Bass, 1977. 

Grunig, J. "Describing Publics in Public Relations: The Case of a Suburban 
Hospital," Journalism Quarterly , Vol. 55 (1978), pn. 109-118. 

Grunig, J., "A New Measure of Public Opinion on Corporate Social 
Responsibility," Academy of Management Journal , Vol. 22 (1979), pp. 
728-764. 

Grunig J. "Communication Behaviors and Attitudes of Environmental 
Publics: fwo Studies." Journalism Monographs . No.81, March, 1983. 

Grunig, J. "Research in the Strategic Management of Public Relations." 
International Public Relatio ns A ssociation Review , in press. 

Grunig, J, and Hunt, T. Managing Public Relations , Holt, Rinehart and 
Winston, New York, NY, 1984. 



ERLC 



81 



88 



Grunig, L. "Activism and Organizational Response: Contemporary Cases of 
Collective Behavior," paper presented to the 15th Annual conference 
of the North American Association for Environmental Education, 
Eugene, OR, September 1986. 

Grunig, L., "Environmentalism and Organizational Response: Contemporary 
Cases of Collective Behavior," paper presented to the Public 
Relations Division, Association for Education in Journalism and Mass 
Communication, Norman, OK, August 1986. 

Hage, J. Theories of Organizations . New York: Wiley Interscience, 1980. 

Hovland, C, Janis, I. and Kelly, H. Communication and Persuasion , Yale 
University Preess, New Haven, CT, 1953. 

Katz, E., Blumler J. and Gurevitch, M. "Utilization of Mass Communication 
by the individual" in Blumler, J.G. and K tz, E. (eds.) The Uses of 
Mass Communications . Beverly Hilk,: Sage Publications, 1974. 

Katz, E. Social Research and Broadcasting , London, BBC, 1977. 

Katz, D. and Kahn, R. The Social Psychology of Organizations . New York: 
Wiley and Sons, 1975. 

Katz, E. and P.E. Lazarsfeld, Person al Influence , Free Press, Glencoe, IL, 
1955. 

Klapper, J. Effects of Mass C ommunication, Free Press, New York, NY, 
1960. 

Lazarsfeld, P., B. Berelson and H. Gaudet, The People's Choice , Duell, 
Sloan and Pearce, New York, NY, 1944. 

Mason, W.S. "Planning for Research Use in Postsecondary Governance and 
Finance." in Proceedings of a Conferenc e on Disseminating Findings 
of Research on Postsecondary G overnance and Finance t Novembe r 21, 
1986 , University of Maryland, College Park. 

McAlister, A. "Anti-smoking Campaigns: Progress in Developing Effective 
Communications," in Rice, R. and Paisley, W. (eds.) Public 
Communication Campaigns . Beverly Hills: Sage, 1981, 91-103. 

McCombs, M. and Shaw, D. "The Agenda Setting Function of Mass Media," 
Public Opinion Quarterly . 36 (1972), pp. 176-87. 

McQuail, T>. Toward a Sociology of Mass Communication , Collier- 
MacMHlan, Londor, 1969. 

McQuail, D. Communication , Longman, London, 1975. 



82 



ERLC 



89 



McQuail, D. Communication Theory , Sage Publications, Beverly Hills, CA,. 



McGuire, W. "The Nature of Attitudes and Attitude Change," in G. Lindsay 
and E. Aronson (eds.), The Handbook of Social Psychology , 2nd ed., 
vol.3, Addison-Wesley, REading, MA, 1969, pp. 136-314. 

lioGuire, W. "Persuasion, Resistance and Attitude Cuange," in I. de soal 
Pool, et al (eds.), Handbook on communication, Rand McNally, 
Chicago, IL, 1973, pp.216-252. 

' McNamara, E. Kurth, T. and Hansen, D. "Communication Efforts to 
Prevent Wildfires," in Rice, R. and Paisley, W. (edsc) Public 
Communication Campaigns . Beverly Hills: Sage, 1981, pp. 143-160. 

Mendelsohn, H. "Some Reasons WHy Information Campaigns Can Succeed." 
Public Opinion Quarterly . 37(1973): pp. 50-61. 

Molnar J. and Rogers, D. "Organizational Effectiveness: An empirical 
comparison of the goal and system resource approaches." The 
Sociological Quarterly . 17(1976), pp. 401-413. 

Rogers, E. "Communication and development: The Passing of a Dominance 
Paradigm," Communication Research , 3: 231-40. 

Rogers, E. and Shoemaker, F. Communication of Innovations . Glencoe: 
Free Press, 1973. 

Schramm, W. "How communication works," in Schramm, W. (ed.) The 
Process and Effects of Mass Communication . Urbana: University of 
Illinois Press, 1954. 

Schramm, W. and Roberts, D. (eds.) The Process and Effects of Mass 
Communication . Urbana: University of Illinois Press, 1971. 

Shannon, C. and Weaver, W. The Mathematical Theory of Communication . 
Urbana: University of Illinois Press, 1949. 

Weiss, C. '" Valuation research , Englewood Cliffs, N.J.: Prentice-Hall, 1972. 

Yuchtman, E. and Seashore, S. "A system resource lpproach to 
organizational effectiveness." American Sociological Review 32(1967), 
pp. 981-902. 



ERLC 



83 

90 



CONFERENCE DISCUSSION 
Ward S. Mason 



9 

ERIC 



84 

91 



CONFERENCE DISCUSSION 



Following the presentation of papers, a general discussion took place 
in which paper presenters, Center staff, OERI representatives, and invited 
guests took part (see Appendix B for a list of participants). What follows 
is a synthesis of the discussion. It is based on written notes; therefore, 
no attempt is made to attribute statements. The document is organized 
by topic rather than by the natural flow of the discussion. 

Dissemination Planning . The conference was concerneu with planning 
a dissemination program. Many of the comments dealt with just how to 
move the planning process forward. Discussants recognized the need to 
plan for dissemination early rather than to wait for research to be 
completed. They suggested that communication planners start with 
constituency needs, working backward to both specific dissemination 
events and to the research agenda. 

Center communication plans should take into account: 

o Short term vs. long term impact; 

o Specification of effects sought, such as changing minds, 
reconceptualizing problems, or defining the range of possible 
solutions; and 

o Types and numbers of people and institutions to be targeted. 

The Center was urged to maintain a degree of flexibility in planning 
for communication. Opportunities for dissemination are often emergent 
rather than anticipated. Successful centers usually have a core set of 
activities that define their identity and have the flexibility to respond to 
unforseen opportunities. In designing specific activities it is always 
important to differentiate the segments of the audience and to choose the 
most appropriate dissemination vehicle for each. However, precise 
targeting is neither possible nor desirable. A certain amount of 
redundancy in messages is essential to good dissemination. 

Professionals indicated that the pr^ss finds newsletters useful. 
However, a good newsletter contains content, not organizational "gossip." 
Participants identified conferences as useful mechanisms for involving 
both researchers and practitioners in dissemination. Conferences focus 
effort and create visibility and synergy. 

Legitimacy . Survival of the Center will depend on establishing its 
legitimacy through a variety of public communication activities. 
Communication mu*** make clear that the Center serves the public 
interest. In doing so, the Center °hould also be responsive to inquiries. 
The Center needs to think through a policy concerning inquiries and 
referrals to other organisation?, such as the J2RIC Clearinghouse on 
Higher Education. Some services might be operated on a cost recovery 
basis* 
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Impact Evaluation and the Problem of Knowledge Use . A rignificant 
tension exists between the need to evaluate the Center's impact and the 
limitations of evaluation research. Appropriate and feasible impact goals 
are essential. Different criteria are needed for measuring different kinds 
of impact. The Center should tailor the evaluation of different 
dissemination activities to different levels on the steps of the 
communication scale (i.e. awareness, diagnosis* consideration of 
alternatives, action, and evaluation). But one should not expect the same 
impact from a newsletter as from a conference. 

According to researchers, there are many kinds of knowledge use. 
In particular, not all uses are intended to achieve a goal. Communication 
that helps policy makers to redefine problems, or to approach th^m in 
different ways, may be an "enlightenment use" that is just as important 
as deriving a specific solution or recommendation, i.e. "instrumental use." 
Nor is all knowledge use rational. Rather, knowledge use is one 
ingredient in the complex interaction of social forces. 

Cooperative Relationships * The Center needs to leverage its scarce 
resources by developing cooperative relationships with other research and 
service organizations. Some activities might be sponsored jointly. 
Careful consideration of the activities and services of other groups can 
lead to a more rational division of labor. 

Working with OERI's regional laboratories may be useful. They are 
concerned with state-wide policy-making at all education levels and aro 
interested in the articulation between pre-collegiate and postsecondary 
education. The FIPSE program is operated by the Department of 
Education and might fund the implementation of some Center products. 

Management . The importance of the dissemination function needs to 
be communicated to the Center's principle investigators. Probably the 
best way of providing some coherence to dissemination would be to 
involve investigators in dissemination planning for their own projects. 

A high status Associate Director of Dissemination is essential in 
order to defend the dissemination function in management councils. A 
high status position requires an appropriately high salary. However, it 
may be difficult to find an individual who could handle both this function 
and conduct research on dissemination. Be prepared to hire a highly 
talented individual whose skills may deviate somewhat from those 
requirements and chen build the division of labor around his or her 
talrnts. 

The Washington metropolitan area has a large number of writers, 
editors, and people with other specialized talents who ar.i available on a 
part-time or contract basis. Utilize this pool of talent. Explore sharing 
personnel or functions with other organizations at the University or at 
other centers. 
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Conclusion . The Center is in its second year of operation. 
Recognizing that dissemination is an essential function of a research 
center, it has wisely chosen to examine research on dissemination and the 
experience of professionals to develop a well-designed dissemination 
program. The ideas generated through other Center planning activities 
should provide a solid foundation for developing specific dissemination 
plans. 
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APPENDIX A 



Attendees at conference planning meeting, August 14, 1986: 

o Salvatore Corrallo, Office of Educational Research and 
Improvement, Department of Education 

o Russell Edgerton, President, American Association of Higher 
Education 

o Elaine El-Khawas, Vice President and Director, Division of 
Policy Analysis and Research, American Council on Education 

o Richard Millard, President, Council on Postsecondary 
Accreditation 

o Ginny Carter Smith, (title), CASE 

o Irving Spitzberg. Director, Council for Liberal Learning, 
Association of American Colleges 

o Barbara Taylor, (title), Association of Governing Boards 

o Robert Berdahl, Acting Execu v .ive Director, National Center for 
Postsecondary Governance and Finance 

o Ward S. Mason, consultant 
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APPENDIX B 



DISSEMINATION MEETING 
November 21, 1986 
9:30 a.m. to 3:30 p.m. 
Center Conference Room 

List of Participants 



Invited Participants 

James Bencivenga, Director 
Information Services Program 

Office of Educational and Research Improvement 
555 New Jersey Avenue, N.W. 
Washington, D.C. 2020c 
(202) 357-6556 

Mike Bowler, President 
Education Writers Association 
Evening Sun 
P.O. Box 1377 
Baltimore, MD 21278 
(301) 332-6019 

Dr. Salvatore Corrallo, Center Liaison 

Office of Educational and Research Improvement 

Room 612 

555 New Jersey Avenue, N.W. 
Washington, D.C. 20208 
(202) 357-6247 

Dr. William N. Dunn 

Professor and Director of the Study 

of Knowledge Use Program 
Graduate School of Public and 

International Affairs 
3J 28 Forbes Quadrangle 
University of Pittsburgh 
Pittsburgh, PA 15260 
(412) 648-7634 

Dr. Jonathan D. Fife, Director 
ERIC Clearinghouse on 

Higher Education 
George Washington University 
One Dupont C ols, N.W. 
Suite 630 

Washington, D.C. 20036 
(202* 296-2597 
632-6212 
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Dr. Ronald G. Havelock, Research Professor 
Center for Productive Use of Technology 
George Mason University, Metro Campus 
3401 North Fairfax Drive 
Suite 322 

Arlington, Virginia 22201 
(703) 841-2686 

John Hollifield, Assistant Director 

Center for Effective Elementary and Middle Schools 

The Johns Hopkins University 

3505 North Charles Street 

Baltimore, Maryland 21218 

(301) 338-7570 

Ms. Mary Jocelyn 
Editor/Public Relations 
NCRIPTAL - Suite 2400 
School of Education 
University of Michigan 
Ann Arbor, MI 48109-1259 
(313) 936-2748 

Ms. Kathleen S. Kelly 
Associate Dean 
College Oi Journalism 
Room 2100 

University of Maryland 
College Park, MD 20742 
(301) 454-4157 

Dr. Ward Mason 

Educational Research Consultant 
10636 Muirfield Drive 
Potomac, MD 20854 
(301) 469-6052 



National Center Staff 

Dr. Richard Chait 
Executive Director 

Dr. Frank Schmidtlein 
Associate Director of Research 

Dr. Toby Milton 
Assistant Director of Operations 

Ms. Martha Gagnon 
Graduate Assistant 
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